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Abstract 

The  acquisition  method  used  to  gain  access  to  new  technologies  can  heavily  influence  whether 
our  war  fighters  have  the  required  tools  to  fight  increasing  and  constantly  shifting  global  threats 
(Kessler  and  others,  2000).  The  purpose  of  this  study  was  to  investigate  the  process  and  results 
of  internal  laboratory  development,  testing  and  fielding  of  small  weapons  systems  as  compared 
to  traditionally  acquired  systems  which  encourage  customer  pull  and  contractor  development. 
This  research  provides  insight  into  how  Air  Force  Research  Laboratory  (AFRL)  personnel 
choose  an  acquisition  method,  retain  personnel  capability,  and  maximize  product  capability. 

This  study  consolidates  the  opinions  of  subject  matter  experts  and  program  managers  through  a 
wide  range  of  interviewees  within  AFRL.  Data  analysis  and  extensive  literature  review  led  to 
conclusions  such  as:  it  is  believed  that  there  is  no  difference  in  war  fighter  capability  and 
delivery  between  internally  and  externally  developed  systems;  and  internal  efforts  provide  better 
personnel  capability,  but  the  current  status  quo  is  sufficient.  It  was  also  revealed  that  a  majority 
of  AFRL  personnel  feel  pressure  to  always  develop  externally.  Recommendations  include  that: 
AFRL  foster  an  environment  where  the  best  acquisition  method  for  the  government  is  chosen; 
AFRL  design  an  internal  contract  plan  to  keep  internal  efforts  on  target;  and  AFRL  invest  in 
further  concurrent  development  efforts. 
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INTERNAL  VERSUS  EXTERNAL  ACQUISITION  FOR  SMALL  WEAPONS  SYSTEMS 


I.  Introduction 


Introduction 

Future  military  surveillance  capability  needs  are  laid  out  within  the  Joint  War  Fighting 
Science  and  Technology  Plan  (JWSTP)  (2006).  These  needs  are  organized  via  defense 
technology  objectives  based  on  criticality.  The  services  have  not  reached  many  of  the  JWSTP’s 
objectives  and,  in  the  aftermath  of  Iraqi  Freedom,  some  traditionally  less  important  objectives 
are  fast  becoming  requirements  as  services  are  conducting  counterterrorism,  training,  and 
policing  efforts  rather  than  waging  war.  In  such  a  fight,  continuous  surveillance  ability  over  a 
large  area  would  allow  better  pinpointing  of  terrorist  activity  and  subsequent  justice.  For 
instance,  after  a  roadside  car  bomb  explosion,  this  surveillance  video  could  be  closely  examined 
to  identify  and  trace  the  individuals  that  planted  the  explosive  device.  Many  different  programs 
in  the  past  few  years  have  attempted  to  accomplish  this  goal  and  achieved  varying  degrees  of 
success.  Examples  of  these  programs  include  the  Pioneer  (PEO,  2004)  and  Satellites 
(Monmonier,  2002), 

Usually,  a  requirement  like  an  advanced  surveillance  system  will  enter  the  Planning 
Programming  and  Budgeting  Execution  System  (PPBS)  process  (DoD  7045.14,  2003)  where, 
over  a  period  of  years,  the  requirement  will  be  planned,  funded,  contracted,  developed,  produced, 
and  sustained.  However,  in  this  case,  over  the  past  two  years,  a  complete  working  system  called 
project  Angel  Fire  was  developed  internally  by  Air  Force  students  in  conjunction  with  the  Air 
Force  Research  Laboratory  (AFRL)  to  address  surveillance  needs  in  our  current  environment. 
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The  Angel  Fire  effort  did  not  start  through  requirements  planning,  nor  does  it  have  a 
program  office  ensuing  its  compatibility  and  supportability.  In  short,  it  is  an  anomaly  within  the 
DoD  as  a  system  that  is  likely  needed,  is  already  complete,  and  was  internally  developed.  The 
concept  of  rapid  internal  development  rather  than  traditional  external  development  and  a  push 
rather  than  strictly  pull  acquisition  system  are  the  focus  of  this  thesis.  Projects  like  Angel  Fire 
and  a  very  few  other  recent  DoD  efforts  demonstrate  successful  application  of  this  methodology. 
Although  internal  development  and  push  concepts  are  nonnal  in  basic  business  principles  and 
application  (Wheelwright  and  Clark,  1994),  they  are  not  prevalent  in  the  DoD  programs  and 
systems. 

Purpose 

The  basic  purpose  of  this  study  is  to  investigate  the  process  and  results  of  internal 
laboratory  development,  testing  and  fielding  of  small  weapons  systems  versus  traditionally 
acquired  systems  which  encourage  customer  pull  and  contractor  development.  The  hypothesis  is 
that  these  methods  are  equally  effective  in  providing  war  fighter  support.  If  this  is  true,  a 
reallocation  of  funding  should  be  accomplished  to  allow  internal  agencies  to  achieve  greater 
participation  in  product  development. 

Assumptions 

In  order  to  investigate  the  purpose  and  test  the  hypothesis,  several  assumptions  are 
necessary.  Each  assumption  will  be  tested  or  investigated  within  this  thesis. 

Assumption  1 :  Current  funding  allocations  (primarily  in  6.2,  6.3  and  6.4)  do  not  allow 
internal  agencies  to  achieve  full  participation  in  product  development. 

Assumption  2:  DoD  has  not  taken  full  advantage  of  push  opportunities. 

Assumption  3:  No  one  acquisition  method  has  proven  itself  better  than  others. 
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Assumption  4:  External  development  is  the  method  of  choice  for  decision  makers 
Research  Question 

With  these  assumptions  in  mind,  the  overall  research  question  for  this  study  is:  Is  there  a 
difference  in  warfighter  capability  and  delivety  between  internally  and  externally  developed 
small  weapons  systems  and,  if  so,  why? 

Component  Questions 

The  overall  research  question  was  broken  down  into  several  investigative  questions  based 
on  a  literature  review  of  concepts  important  to  outsourcing. 

-  Which  involves  less  cost? 

•  Will  concurrent  efforts  reduce  cost? 

-  Which  produces  better  capability? 

•  Can  internal  be  done? 

•  Does  one  produce  a  better  customer  interface? 

•  What  defines  personnel  capability? 

•  Does  one  produce  better  personnel  capability? 

-  Which  provides  faster  delivery? 

•  Are  there  process  problems  in  either  acquisition  method 

-  Are  personnel  given  any  guidance  on  the  issue? 

Definitions  and  Concepts 

Before  attempting  to  answer  the  research  or  component  questions,  some  basic  definitions 
and  concepts  are  required  to  ensure  the  ideas  presented  can  be  fully  understood.  The  most 
suitable  method  for  presentation  is  via  a  table,  as  shown  in  Table  1. 
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Table  1  Definitions  and  Concepts 


Term  or  Concept 

Description 

Internal 

Development 

Any  project  which  the  DoD  leads,  directs,  performs  research  on,  and 
retains  all  rights  to.  This  can  include  contractors  who  are  part  of  the 
government’s  internal  team. 

External 
Development  or 
Contractor 
Development 

Projects  which  are  completely  or  almost  completely  outsourced.  The 
government  may  retain  oversight  and  input,  but  the  day  to  day  activities 
are  directed  by  the  contractor  and  product  rights  normally  remain  with 
the  contractor. 

PPBS 

Planning  Programming  and  Budgeting  Execution  System.  The  process 
developed  by  former  secretary  of  defense  Robert  McNamara  whereby 
the  defense  department  plans  and  allocates  resources  for  systems 
through  requirements  planning  years  in  advance  of  acquisition. 

Small  Weapon 
System 

Mid-range  dollar  value  acquisitions  from  approximately  the  UAV  level 
down  to  software  that  is  acquired  via  development 

Traditional 

Acquisition 

Development,  production,  and  sustainment  via  a  contract  effort. 

Concurrent 

Acquisition 

An  internal  development  effort  being  performed  simultaneously  as  one 
(or  more)  external  projects  developing  the  same  product.  These  can 
either  be  competitive,  where  internal  and  external  teams  do  not  typically 
share  results  until  the  product’s  completion,  or  collaborative,  in  which 
internal  and  external  teams  regularly  share  developments. 

Requirements 
Pull  System 

An  acquisition  concept  where  the  end  user  dictates  specific 
requirements  and  the  system  fills  the  need. 

Requirements 
Push  System 

An  acquisition  concept  where  research  and  development  discovers  a 
new  product  or  capability  and  develops  it  for  users  (who  may  not 
realize  the  full  extent  of  its  abilities). 

Development 

Funnel 

A  graphical  description  of  the  process  of  moving  ideas  from 
investigations  though  development  to  production.  Figure  1  is  a 
presentation  of  this. 

Fuzzy  Front 

End 

A  term  to  describe  the  investigations  at  the  beginning  of  the 
development  funnel.  The  process  of  conducting  basic  research  and 
recognizing  potential  in  a  discovery  to  fill  a  customer  need. 
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Continuation  of  Table  1  Definitions  and  Concepts 


Term  or  Concept 

Description 

TRL 

Technology  Readiness  Level.  A  measure  of  technology  maturity  used 
by  DoD  Program  Managers  (PMs)  and  laboratory  employees  to  make 
determinations  concerning  the  readiness  of  a  technology  for 
development  or  use  in  the  field.  Figure  2  provides  a  graphical  depiction 
and  definitions  for  each  level. 

Technology 
Designation  or 
Funding  Stream 

Money  is  broken  into  numerical  technology  designations  sometimes 
called  “pots”  or  “funding  streams”  according  to  the  level  of  technology 
the  money  is  intended  to  support.  The  streams  of  primary  interest  in 
this  thesis  are  6.2  (applied  research),  6.3  (advanced  technology 
development),  and  6.4  (demonstration  and  validation).  These  are 
covered  more  in-depth  in  later  chapters,  but  should  roughly  correspond 
to  TRL  level  as  depicted  in  Figure  3. 

War  Fighter, 
User,  or 
Customer 

Those  who  request  technologies  or  have  needs  that  the  acquisition 
system  must  fulfill. 

AFOSR,  VA, 
DE,  HE,  IF, 

ML,  MN,  PR, 
SN,  VS 

These  are  the  office  symbols  for  the  ten  AFRL  directorates  referenced 
in  this  research.  They  are: 

AFOSR  -  Air  Force  Office  of  Scientific  Research,  VA  -  Air  Vehicles, 

DE  -  Directed  Energy,  HE  -  Human  Effectiveness,  IF  -  Infonnation, 

ML  -  Materials  and  Manufacturing,  MN  -  Munitions,  PR  -  Propulsion, 
SN  -  Sensors,  VS  -  Space  Vehicles 

Investigations  Development  Shpping 

Products 


Figure  1  Wheelwight  and  Clark  Technology  Development  Funnel 
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System  Test,  Launch 
&  Operations 


System/Subsystem 

Development 


Technology 

Demonstration 


Technology 

Development 


Research  to  Prove 
Feasibility 


TRL  9 

TRL  8 

TRL  7 


TRL  4 

TRL  3 


Basic  Technology 
Research 


Actual  system  flight  proven  through 
successful  missions  operations 

Actual  system  completed  and  flight 
qualified  through  test  and 
demonstration 

System  prototype  demonstration  in  a 
operational  environment 

System/Subsystem  model  or 
prototype  demonstration  in  a  relevant 
environment 

Component  and/or  breadboard 
validation  in  relevant  environment 

Component  and/or  breadboard 
validation  in  laboratory  environment 

Analytical  and  experimental  critical 
function  and/or  characteristic  proof- 
of-concept 

Technology  concept  and/or 
application  formulated 

Basic  principles  observed  and 
reported 


Figure  2  TRL  Maturity  Levels 
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Figure  3  Relevant  Funding  Streams  Mapped  to  TRLs 
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Background  and  Problem  Importance 


If  there  is  no  difference  in  war  tighter  capability  and  delivery  between  internally  and 
externally  developed  systems  then  DoD  should  be  taking  advantage  of  internal  product 
development  or  technology  push.  However,  contractors  are  consistently  selected  to  perform 
advanced  research  and  prototype  construction.  Some  experts  believe  an  internally  developed 
system  would  certainly  have  advantages  over  a  contractor  developed  one.  These  include:  patent 
rights  which  allows  the  DoD  to  build  the  weapon  itself  or  contract  to  the  lowest  bidder  thereby 
saving  money,  internal  personnel  development  and  knowledge  which  allows  for  rapid  changes  or 
modifications  to  the  system  should  a  need  arise  from  the  war  fighter,  direct  links  with  the  war 
fighter  to  develop  and  test  exactly  what  they  request,  and  a  reduced  time  to  field  by  removing  the 
bidding,  contract  negotiation,  and  contractor  protests  which  waste  valuable  time  but  are  inherent 
to  the  contracting  system  (Allen,  1995).  By  diverting  some  6.3  and  6.4  funds  to  the  Air  Force 
schools  and  research  laboratories  with  the  mandate  to  develop  basic  technologies  into  working 
weapons  or  develop  a  specific  weapon,  the  government  could  save  money  and  more  quickly 
develop  tools  for  the  field.  Of  course,  not  all  money  will  lead  to  productive  weapon  systems,  but 
it  is  better  to  keep  all  options  open. 

Scope 

Preliminary  research  revealed  that  internal  development  projects  can  be  successful.  This 
thesis  does  not  imply  that  a  large  scale,  highly  technical  and  complex  project  (such  as  an  F-22) 
could  be  developed  through  anything  but  a  joint  effort  between  industry  and  government,  but 
rather  that  projects  of  smaller  scope  can  easily  be  implemented  and  prototyped  within  DoD 
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laboratories.  These  include  mid-range  dollar  value  acquisitions  from  approximately  the  UAV 
level  down  to  software  that  is  acquired  via  development  rather  than  commercial  off  the  shelf 
(COTS).  It  does  not  include  government  impact  card  type  purchases. 

Examples  range  from  software  only  products  to  larger  projects  like  Angel  Fire  detailed 
above.  Some  other  recent  initiatives  include  the  “friendly  marking  device”  (a  type  of  radar  flare 
to  mark  friendly  forces  which  was  rapidly  developed  by  an  internal  team),  the  “vehicle  stopper 
program”  (two  internal  efforts  which  produced  four  prototypes  for  a  better  vehicle  barricade  for 
security  police)  (Kleiman,  2006),  and  several  software  products. 

Due  to  time  and  resources,  this  thesis  will  be  only  a  pilot  study  and  will  involve  only  Air 
Force  Research  Faboratories  (AFRF).  The  author  will  use  case  study  methodology  and  an 
interview  process  to  explore  the  issues  related  to  in-house  or  contracting  development  efforts. 
This  methodology  provides  easy  access  to  interviewees  and  a  research  product  that  can  be 
operationalized  to  the  Air  Force,  and  possibly  to  DoD  laboratories.  More  importantly,  it 
provides  preliminary  data  for  more  in-depth  research  or  a  basis  to  incorporate  new  initiatives 
within  the  laboratories. 

Summary 

The  basic  purpose  of  this  study  is  to  explore  the  possibility  of  internal  laboratory 
development,  testing,  and  fielding  versus  the  traditional  system  which  encourages  customer  pull 
and  contractor  development.  Assumptions  were  made  based  on  literature  reviews  and 
preliminary  discussions  with  laboratory  personnel.  Areas  important  to  outsourcing  were  taken 
from  commercial  and  government  information  sources  and  used  to  develop  component  research 
questions.  The  scope,  importance,  and  definitions  important  to  this  study  were  also  reviewed. 
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II.  Literature  Review 


Introduction 

Numerous  methods  for  conducting  acquisitions  and  contracting  efforts  exist  for  both  the 
government  and  the  civilian  sector.  This  chapter  will  delve  into  that  infonnation  to  describe  and, 
in  one  case,  defend  each  of  the  four  assumptions  made  in  chapter  one.  It  will  also  explore  areas 
the  literature  ties  to  outsourcing  decisions,  which  will  become  the  basis  for  the  interview 
instrument  utilized  to  determine  if  a  difference  exists  in  war  fighter  capability  and  delivery 
between  internally  and  externally  developed  small  weapons  systems. 

Assumption  1:  Current  funding  allocations  (primarily  in  6.2,  6.3  and  6.4)  do  not  allow 
internal  agencies  to  achieve  full  participation  in  product  development. 

The  highly  formalized  system  in  the  DoD  today  may  prevent  performing  all  the 
developmental  steps  usually  undertaken  in  the  commercial  world,  thereby  limiting  options.  For 
example,  the  “fuzzy  front  end”  and  developmental  funnel  has  been  replaced  with  regimented 
bureaucratic  requirements  in  order  to  ensure  meaningful  use  of  taxpayer  dollars  (Khurana  and 
Rosenthal,  1997).  In  commercial  business,  the  fuzzy  front  end  refers  to  the  creative  or  basic 
engineering  process.  While  the  DoD  funds  this  and  even  employs  scientists,  McNamara’s  PPBS 
system  keeps  the  funding  streams  separate  and  divided  into  distinct  codes  (Department  of 
Defense,  2003).  These  codes  are  6.1  (basic  research),  6.2  (applied  research),  6.3  (advanced 
technology  development),  6.4  (demonstration  and  validation),  6.5  (engineering  and 
manufacturing),  6.6  (management  support),  and  6.7  (operational  systems  development).  These 
are  provided  in  Table  2  for  ease  of  viewing. 
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Table  2  Technology  Designations 


Community 

Numerical 

Designation 

Category 

Science  and 

6.1 

Basic  Research 

Technology 

(3600) 

6.2 

Applied  Research 

6.3 

Advanced  Technology  Development 

Acquisition 

6.4 

Demonstration  and  Validation 

(3400) 

6.5 

Engineering  and  Manufacturing  Development 

6.6 

Management  Support 

6.7 

Operational  Systems  Development 

At  first  glance,  these  codes  seem  to  coincide  with  the  lifecycle  development  of  a  product. 
In  fact,  Wheelwright  and  Clark  developed  a  synopsis  of  the  commercial  industry  development 
funnel  and  model  that  can  be  rather  easily  pictured  side-by-side  with  DoD’s  model  in  Table  3 
(1992). 

Table  3  Commercial  Funnel  versus  PPBS 


Commercial  Development  Funnel _ DoD  Technology  Designation  ($$) 


Knowledge  Acquisition 

(6.1)  Basic  Research 

Concept  Investigation 

(6.2)  Applied  Research 

Basic  Design 

(6.3)  Advanced  Technology  Development 

Prototype 

(6.4)  Demonstration  and  Validation 

Pilot  Production 

(6.5)  Engineering  and  Manufacturing  Development 

Manufacturing  Ramp-Up 

(6.6  and  6.7)  Management  Support  and  Operational 
Sciences  Development 
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As  Table  3  demonstrates,  the  DoD  funding  streams  do,  in  fact,  line  up  with  Wheelwright 
and  Clark’s  developmental  funnel  for  Knowledge  Acquisition,  Concept  Investigation,  Basic 
Design,  Prototype,  Pilot  Production,  and  Manufacturing  Ramp-up.  However,  two  main 
differences  between  the  two  processes  exist.  First,  rather  than  being  a  continuous  funnel,  DoD 
separates  funding  according  to  6.1,  6.2,  6.3,  etc.  This  division  of  funding  streams  can  cause 
issues  when  attempting  to  advance  or  transition  technology  from  one  level  to  the  next.  Second, 
unlike  a  commercial  business  which  contracts  out  only  as  needed,  DoD  tends  to  contract  out  the 
majority  of  work  in  6.2,  6.3  and  6.4  (indicated  by  bold  in  Table  3).  In  fact,  Air  Force 
laboratories  are  actively  encourage  to  contract  out  eighty  percent  or  more  of  the  research  they 
conduct  (Masqulier,  2006).  The  acquisition  system  is  inherently  designed  to  start  with  a  pull 
requirement,  develop  a  program  office,  and  hire  a  contractor  to  perform  6.2,  6.3,  and  6.4 
research,  development  and  manufacturing  (Guttman,  2003).  Once  the  weapon  system  is  brought 
on  line,  the  system  might  transition  to  a  depot  for  lifecycle  management  and  support  or  these 
requirements  may  stay  with  the  contractor. 

Assumption  2:  DoD  has  not  taken  full  advantage  of  push  opportunities. 

In  the  commercial  world,  basic  and  applied  research  serves  one  of  two  purposes:  it  either 
facilitates  a  technology  pull  or  is  being  performed  with  the  goal  of  implementing  a  technology 
push  (Khurana  and  Rosenthal,  1997).  For  the  commercial  world,  a  technology  pull  concept 
implies  someone,  possibly  the  customer  or  marketing,  has  requested  that  a  product  and 
technology  development  be  engineered  to  support  that  product  (“Defense  Acquisition 
Handbook”,  2005)  In  a  technology  push  concept,  a  scientist  performs  basic  research  to  develop 
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an  item  that  might  be  useful  to  a  product  line  or  something  that  can  be  taken  directly  to  market. 
These  systems  are  illustrated  in  Figure  4  (“Defense  Acquisition  Handbook”,  2005) 


Figure  4  System  Processes 

Under  the  government’s  current  financial  and  budgeting  system,  none  of  the  linkages 
between  product  and  research  demonstrated  in  the  commercial  world  exist.  Universities  and 
laboratories  receive  basic  and  applied  research  funding  with  the  belief  that  research  will  lead  to 
new  technologies  which  can  be  utilized  in  the  future  (House  Committee  on  Science,  1999).  No 
end  product  drives  the  research.  Recent  ideas,  such  as  Advanced  Concept  Technology 
Demonstrations  (ACTDs),  and  plans,  such  as  the  Joint  Warfighting  Science  and  Technology  Plan 
(2006),  have  attempted  to  bridge  the  gap  between  internal  ideas  and  formal  product  development 
by  providing  dialogue  between  researcher  and  war  fighter  or  providing  general  long-term 
capabilities  requirements  (Riley,  2003).  ACTD  provides  the  closest  parallel  to  a  normal  push 
concept.  In  an  ACTD,  system  prototypes  are  demonstrated  to  potential  users  who  detennine  if 
the  product  is  needed  and,  if  it  is,  usually  acquire  it  or  fund  it  though  the  normal  PPBS  system 
(“Joint  Capability,”  2004;  DoD,  2002).  The  difference  in  ACTD  and  the  push  system  shown  in 
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the  above  figure  is  that,  even  if  further  development  is  required,  a  prototype  has  already  been 
developed;  this  effectively  puts  need  after  development.  This  process  does  not  link  the  research 
to  a  specific  product  or  negate  the  necessity  to  enter  a  pull  process  in  order  to  actually  fund  a 
development  project.  Instead,  where  products  (weapon  systems)  are  concerned,  the  DoD 
operates  on  a  PPBS  driven  technology  pull  concept  that  relies  on  proven  technologies  or  proven 
contractors  and  requires  planning  acquisitions  five  years  in  advance  (DoD,  2003).  This  implies 
that  either  the  technology  is  proven  before  it  enters  the  process  for  funding  or  it  is  supported  by  a 
contractor  who  has  proven  capability  for  advanced  technology  development. 

Wheelwright  and  Clark  developed  a  synopsis  of  the  commercial  industry  development 
funnel  and  model  that  can  be  viewed  side-by-side  with  DoD’s  model,  as  shown  in  Figure  5 
(1992).  The  top  and  bold  positions  of  the  figure  encompass  Wheelwright  and  Clark’s  model 
while  the  lower  portions  are  the  DoD  equivalent  processes.  Note  that  the  DoD  does  not  have  a 
requirements  push  process,  but  relies  almost  exclusively  on  defined  requirements  rather  than 
reviewing  what  technology  can  provide. 

In  a  normal  business,  technology  assessment  and  forecasting  and  market  assessment  and 
forecasting  are  reviewed  together  to  plan  the  most  likely  products  to  succeed.  This  allows 
civilian  industry  to  avoid  mistakes  often  made  by  the  government  (reaching  for  technology  that 
is  not  yet  available)  and  to  more  easily  transition  technology  to  the  end  user  (Wissler,  2006:  7). 
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Development  Model 


Technology  Assessment 
and  Forecasting 


Technology  Push 


Market  Assessment 
and  Forecasting 

Requirements 

Planning/Wants/Technology  Pull 


Development  Goals  and  Objectives 

Joint  Warfighter  Science  and  Technology  Plan  (JWSTP) 

Aggregate  Project  Plan 

PPBS/ Acquisition  System/Contracting 

Project  Management  and  Execution 

SPO/Project  Manager/Contractor 

Post  Project  Learning  and  Improvement 

Audit/GAO 


Figure  5  Development  Models 

Assumption  3:  No  one  acquisition  method  has  proven  itself  better  than  others. 

Unlike  previous  assumptions  which  will  be  addressed  in  the  results  of  this  research  effort, 
this  assumption  relies  on  historic  fact  so  it  must  be  proven  or  disproven  within  the  literature 
review.  This  involves  a  review  of  the  arguments,  solutions,  and  problems  the  acquisition  process 
has  endured  in  the  past  to  illustrate  that  all  modern  acquisition  methods,  including  internal 
development,  have  been  attempted  in  the  past,  but  none  have  proven  best.  At  the  heart  of  this 
assumption  is  the  ever-present  movement  from  integration  or  vertical  integration  to 
disintegration  or  horizontal  integration  and  back  in  a  continuous  loop  or  helix  with  no  one 
method  ever  proving  perfect  (Fine,  1998).  This  helix  is  present  both  in  industry  and  DoD.  To 
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illustrate  that  no  one  acquisition  method  has  proven  better  than  another  in  DoD,  this  portion 
relies  heavily  on  two  authors,  James  Nagel  and  Wilbur  Jones,  who  have  written  much  on  military 
acquisitions  history.  It  is  divided  into  two  primary  phases:  a  review  of  early  acquisition  history 
and  then  a  comparison  of  these  methods  to  modern  thought. 

Early  United  States  Acquisition  History  (1775-1975) 

James  Nagel’s  A  History  of  Government  Contracting  (1992)  and  Wilbur  Jones’  Arming 
the  Eagle  (1999)  provide  excellent  overviews  of  contracting  and  acquisition  roots  in  the  US.  An 
argument  could  be  made  that  the  basic  issues  they  review  apply  to  any  country  or  entity 
performing  a  contracting  process;  however,  for  the  purposes  of  this  thesis  and  its  scope,  these 
two  books  fonn  an  adequate  baseline  without  a  need  for  international  implications.  Nagle 
attributes  the  issues  in  his  two-hundred  year  history  of  government  contracting  and  acquisition 
process  changes  to  five  recurring  themes  which,  he  argues,  have  plagued  the  US  since  the 
Revolutionary  War.  These  include  the  military-industrial  complex,  profiteering,  government  vs. 
contractor  sourcing,  ethics,  and  competition. 

Nagle  states  that  the  country’s  opinion  of  the  military-industrial  complex  rotates  in  three 
phases  each  time  a  great  emergency  faces  us:  first,  awe  at  the  industrial  strength  and  America’s 
ability  to  conquer  the  task;  second,  complaints  about  the  speed,  efficiency,  quality  and  cost;  and 
finally,  disgust  at  the  final  price  and  profits  (Nagle,  1992:  4).  This  opinion  rotation  occurred  in 
both  great  civilian  undertakings,  like  the  Hoover  dam,  and  military  undertakings,  like  the  F-22 
today.  In  particular,  the  American  war  machine  has  entered  this  cycle  during  each  war,  resulting 
in  a  quick  demobilization  after  every  war  until  the  Korean  War  (Nagle,  2002:57).  The  result  of 
this  repeated  demobilization  was  a  difference  in  contracting  and  acquisition  needs  in  peacetime 
and  wartime.  The  led  to  contracting  requirements  that  were  sufficient  during  peace,  but  were  too 
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slow  for  war  (Nagle,  2002:58).  In  the  1800s,  a  practice  was  introduced  which  allowed  small 
contracts  for  weapons  and  ammunition  production  to  be  granted  to  civilian  industry  in  order  to 
ensure  their  proficiency.  By  1930,  this  became  the  standard  procedure  of  maintaining  industrial 
capability  without  maintaining  the  large  industrial  complex  (Jones,  1999:  221).  However,  new 
aircraft  technology  contractors  in  the  1920s  were  given  “shared  risk  and  cost”  rather  than  “fixed 
price”  contracts.  At  times,  these  contractors  were  given  full  control  of  their  projects  (Jones, 
1999:221).  Although  this  close-knit  relationship  led  to  arguably  excellent  results,  it  fell  under 
heavy  congressional  scrutiny  during  the  depression.  Arguments  arose  that  the  aircraft  industry 
had  made  obscene  profits  at  taxpayer  expense;  this  led  to  very  stringent  cost  and  profit  margin 
control  for  the  next  twenty  years  (Jones,  1999:222).  Until  these  controls  were  loosened  in  the 
First  War  Powers  Act  of  1941,  the  modern  military-industrial  complex  did  not  begin  to  arise 
(Jones,  1999:  262).  With  the  Second  War  Powers  Act,  which  included  the  government’s  ability 
to  audit  private  contractor  books  and  records,  the  modern  acquisition  relationship  was  realized 
(Jones,  1999:  265). 

As  early  as  1777,  “profiteering”  began  to  occur  in  the  US  government  acquisition 
process.  In  this  timeframe,  farmers,  merchants,  and  those  supplying  the  Anny  took  advantage  of 
the  increased  demand  to  increase  price  and  profit  (Nagle,  1992:  45).  In  an  attempt  to  prevent 
this,  the  government  shifted  from  buying  goods  on  the  open  market  to  long  term  contracting  with 
a  single  supplier  to  support  all  Anny  needs  in  a  particular  category  for  a  given  geographic 
region.  This  direct  support  shared  sustainment  style  contracting  system  (much  like  the  F-22 
parts  contract  today)  was  put  in  place  in  1781.  To  the  detriment  of  the  troops,  contractors  took 
advantage  of  the  government  almost  immediately  (Nagle,  1992:  54).  As  George  Washington 
noted  to  Comfort  Sands,  who  was  supposed  to  be  supplying  the  Army: 
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“Why  Sir  are  the  Troops  without  Provisions?  Why  are  the  deposits  which  have  so  often, 
and  so  long  ago  been  required  by  General  Health,  and  pressed  by  myself,  neglected? 

Why  do  you  so  pertinaciously  adhere  to  all  those  parts  of  the  Contracts  as  are  promotive 
of  your  own  Interest  and  convenience  .  .  .  and  at  the  same  time  disregard  the  most 
essential  claims  of  the  public;  thereby  hazarding  the  dissolution  of  the  Anny  and  risking 
the  loss  of  the  most  important  Post  in  America?”  (Nagle,  1992:  54) 

In  fact,  profiteering  was  far  worse  than  a  few  contractors  bending  the  words  of  the  contract  to 

their  benefit;  merchants  were  colluding  to  increase  prices  and  buying  stock  in  each  other  to 

destroy  any  real  competition  (Nagle,  1992:  55).  Between  1796  and  1798,  six  frigates  were 

promised  for  $688,000,  but  cost  overruns  caused  only  three  ships  to  be  built  at  a  final  cost  of 

over  $1.2M.  Again,  rampant  complaints  of  profiteering  ensued,  but  the  three  ships  built  were  the 

“most  magnificent  ships  afloat”  at  that  time  (Nagle,  1992:  71).  At  times,  the  profiteering 

accusation  was  wholly  unfounded;  rather,  it  was  due  to  increasing  technology  cost.  For  example, 

in  the  Navy,  between  1930  and  1940,  new  technologies  like  sonar  and  radio  swelled  shipboard 

electronics  costs  from  $17,000  to  over  $100,000  (Jones,  1999:  225).  No  solution  to  profiteering 

has  been  found,  since  congressional  investigation  and  complaint  usually  come  after  project 

completion  and  very  few  sanctions  have  ever  occurred. 

The  arguments  for  government  versus  contractor  sourcing  have  also  been  around  since 

the  early  days  of  the  country.  In  the  first  days  of  the  Revolutionary  War  in  1775,  congress 

established  the  Army  contracting  system  relying  entirely  upon  the  government  to  procure,  store 

and  transport  goods  (Nagle,  1992:  24).  By  1776,  arguments  arose  that  contractors  could  provide 

a  better  system  at  lower  cost  (Nagle,  1992:  32).  In  1781,  under  Robert  Morris  and  after 

numerous  government  provided  supply  failures,  scandals,  and  fraud  accusations,  contracts  were 

introduced  for  private  companies  to  supply  the  Anny  (Nagle,  1992:  51).  Unfortunately,  by  1782, 

complaints  arose  about  contractor  profiteering  (Nagle,  1992:  54).  In  response  to  this  and 

numerous  subcontracts,  which  confused  who  was  responsible  for  low  quality  supplies,  Morris 
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appointed  an  inspector  of  contracts  and  inserted  contract  clauses  for  inspection  (Nagle,  1992: 

54).  Unfortunately,  this  did  not  meet  its  intended  goal,  and  by  the  end  of  1782  the  contracting 
system  had  failed  to  provide  as  promised.  As  a  result,  many  commanders  contracted  directly  to 
supply  their  men  (Nagle,  1992:  55).  Fortunately,  the  Revolutionary  War  soon  ended  and  these 
problems  did  not  arise  again  until  1794.  In  his  battles  with  Native  Americans,  Major  General 
Wayne  once  again  pointed  out  the  “absolute  necessity  of  some  effectual  &  Certain  mode  of 
supplying  the  army  than  that  of  private  contract.”  (Nagle,  1992)  This  led  to  additional  attempts 
to  refine  the  contracting  process  which  resulted  in  varying  degrees  of  success  (Nagle,  1992:  69). 
By  the  early  1800’s,  a  combination  system  of  government  and  contractor  production  and  supply 
of  small  anns  existed  that  existed  in  the  US  through  the  War  of  1812.  However,  sentiments 
continued  to  shift,  first,  in  favor  of  national  armories,  and  then,  after  the  Mexican  War,  back  to 
exclusive  dependence  on  contract.  In  1850,  Jefferson  Davis  defeated  an  attempt  by  congress  to 
completely  privatize  the  industry  arguing  that  both  were  needed  for  the  best  possible  system 
(Nagle,  1992:  89).  Davis  argued  that  national  armories  were  readily  available,  less  expensive, 
and  established  a  standard  for  price,  while  private  manufacturers  innovate  in  both  materials  to 
increase  firepower  and  methods  to  lower  production  cost  (Nagle,  1992:  89). 

Ethical  issues  also  caused  problems  within  the  acquisition  process.  At  times  throughout 
the  early  days,  suppliers  and  commissary  generals  were  paid  via  commission  based  on  the 
amount  of  money  that  went  through  their  hands;  this  followed  traditional  European  practice.  In 
fact,  it  was  commonplace  to  commingle  ones  own  money  with  the  govermnent  money  one  was 
managing.  Often,  government  money  would  be  used  for  private  ventures  and  vice  versa;  this  led 
to  numerous  issues.  For  one,  congress  caught  commissaries  significantly  increasing  prices  to 
make  more  commission  profit  (Nagle,  1992:  34).  Congress  attempted  early  on  to  go  to  a  salary 
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based  system;  however,  this  failed  because  Congress  could  not  pay  an  enticeable  salary  without 
raising  the  pay  of  others  in  similar  ranked  positions  (Nagle,  1992:  40).  Instead,  congress 
attempted  to  combat  fraud  by  regulation,  which  produced  nearly  disastrous  results.  Many 
capable  managers  left  rather  than  deal  with  the  requirements  and  the  regulations  which  sacrificed 
efficiency  at  a  time  when  swift  contracting  was  desperately  needed  (Nagle,  1992:  38).  In  1779, 
congress  attempted  to  demand  line  item  accounting  for  purchases,  arguing  that  the  money 
flowing  through  commissary  and  quartermaster  hands  was  not  producing  equitable  results. 

These  calls  were  met  with  excuses  and  extensions  or  were  just  flatly  ignored  (Nagle,  1992:  41). 
In  other  cases,  ethical  people  were  falsely  accused  of  defrauding  the  government.  In  1798,  Eli 
Whitney  won  a  contract  to  deliver  muskets  to  the  Revolutionary  Army  within  two  years.  It 
actually  required  over  1 1  years  to  deliver  all  of  the  muskets.  Despite  accusations  of  fraud  and 
profiteering,  the  government  continued  to  pay  what  would  be  considered  incentive  or  award  fees 
in  full  and  advances  on  the  final  sum,  and,  in  the  end,  Eli’s  technology  of  interchangeable  parts 
proved  invaluable.  Several  other  contractor  bids  were  accepted  to  get  the  total  number  of 
muskets  required;  many  of  these  contractors  were  also  granted  extensions,  received  incentives 
and  advances,  and  never  produced  anything,  which  is  a  much  worse  example  of  defrauding  the 
government  (Nagel,  1992:  84). 

Finally,  in  the  area  of  competition,  Congress  has  always  been  partial  to  a  competitive  bid 
lowest-cost  fixed-price  contract.  In  Nagle’s  opinion,  this  is  often  the  least  efficient  acquisition 
vehicle  and  typically  leads  to  a  last  minute  need  to  suspend  the  rules  and  rush  to  prepare  for  war. 
As  early  as  1781,  congress  approved  sealed  bid  contracting.  Shortly  thereafter,  sealed  bid  with 
negotiation  with  “responsibility”  (prior  proven  ability  to  meet  contract  requirements),  with  cost- 
benefit  calculations,  and  with  other  variations  on  the  standard  sealed  bid/low  price  were 
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introduced  (Nagle,  1992:  52).  By  1799,  every  style  of  contracting  vehicle  currently  in  use  had 
been  attempted  with  varying  degrees  of  success.  These  include  cost  plus  (at  the  time  called  cost 
plus  commission)  in  yet  another  attempt  to  keep  the  troops  adequately  fed  by  ensuring  the 
contractor  made  a  profit,  and  government  furnished  materials  which  attempted  to  keep  the  supply 
of  uniforms  uninterrupted  by  pre-buying  the  cloth  and  then  furnishing  it  to  contracted  tailors 
(Nagle,  1992:  73).  From  1805  to  about  1840,  the  government  even  considered  private  armories, 
which  were  considered  part  of  the  public  supply  of  arms  and  resulted  in  almost  zero  competition. 
In  other  words,  long-term  contracts  were  awarded  that  were  continually  renewed  (Nagle,  1992: 
87).  This  gradually  evolved  to  re -bidding  with  renewal  if  performance  was  satisfactory  and  the 
price  bid  was  as  low  as  any  other  bid  (Nagle,  1992:  88).  Of  particular  note  regarding  contracts, 
is  the  recurring  theme  of  buying  or  acquiring  patent  rights.  For  instance  in  1816,  a  contract  with 
Asa  Waters  for  5000  muskets  at  $14  each  included  the  right  of  the  United  States  to  use  his 
patented  barrel  welding  technology  in  its  armories  as  well  as  the  right  to  extend  this  to  any 
contractor  making  anns  for  the  government  (Nagle,  1992:  86).  In  1917,  a  combined  patent  pool 
in  the  form  of  the  Manufacturers’  Aircraft  Association  served  the  same  need  (Jones,  1999:  202). 
By  acquiring  patent  rights,  the  government  could  assure  fair  competition  and  generally  lower  bid 
prices.  By  1918,  acquisition  regulations  had  been  formalized  partly  due  to  the  need  to  ensure 
competition.  This  five  step  process  included  specification  preparation,  advertising  for  bids, 
securing  guarantee  bonds,  signing  the  contract,  and  executing  it  (Jones,  1999:  180).  Except  for 
securing  guarantee  bonds,  the  steps  were  similar  to  the  methods  used  today  (GSA,  2005)  and 
have  made  the  efforts  to  produce  new  weapons  extremely  slow.  For  example,  it  took  17  years  to 
develop  the  Ml  rifle  (Jones,  1999:  220).  By  1943,  the  steps  had  increased  to  15;  the  need  for 
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prior  year  planning  became  evident  to  proceed  though  the  steps  that  were  meant  to  ensure  fair 
and  equitable  competition  (Jones,  1999:  268). 

Attempts  at  streamlining  the  system  were  made  throughout  the  late  1940s  and  1950s. 

This  strengthened  the  Defense  Secretary’s  position  as  well  as  consolidated  some  classifications 
and  committees  but  did  little  to  change  the  system  until  1961  with  the  appointment  of  Robert 
McNamara  (Jones,  1999:325).  McNamara  created  the  Office  of  Systems  Analysis,  Defense 
Contract  Audit  Agency,  Defense  Contact  Administrative  Service,  and  the  Defense  Supply 
Agency.  McNamara  also  initiated  several  programs  aimed  to  increase  competition  and 
incentivize  contracting;  some  of  these  programs  are  still  being  used  today  while  others  have  been 
abandoned  (Jones,  1999:  329).  Most  notably,  McNamara  created  the  Planning,  Programming, 
and  Budgeting  System  which  is  still  in  use  today  and  was  originally  meant  to  bridge  the  gap 
between  the  disjointed  planning  and  programming  (Jones,  1999:  328). 

History  (1775-1975)  vs.  Modern  Thought  (1975-present) 

Modern  government  acquisition  has  been  called  a  sea  of  paperwork  (Nagle,  1992:  505) 
culminating  from  thousands  of  decisions  made  by  thousands  of  people  rooted  in  special  interest, 
success,  failure,  and  conflicting  ideologies  (Nagle,  1992:  519).  The  detailed,  many  volume 
manuals  regulating  modem  acquisition  includes  the  Federal  Acquisition  Regulation  (FAR),  DoD 
7045.14  Planning  Programming,  and  Budgeting  System  (2003),  DoD  7045-H  Planning 
Programming,  and  Budgeting  Handbook  (2004),  as  well  as  many  others.  In  fact,  due  to  the 
confusing  nature  of  the  numerous  regulations,  texts  have  been  written  in  an  effort  to  condense 
the  infonnation  into  an  understandable  and  usable  form.  Some  of  the  better  resources  in  this  area 
include  Arnavas’  Government  Contracting  Guidebook  (2001),  the  Air  Force  Institute  of 
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Technology  Defense  Acquisition  Handbook  (2005),  and  the  Department  of  Defense  Acquisition 
Strategy  Guide  (1994). 

None  of  these  vast  regulations  seem  to  have  solved  overarching  problems  or  quieted  the 
continuous  debate  about  the  proper  acquisition  methodology.  In  fact,  the  issues  encountered  in 
modem  academic  papers  are  surprisingly  similar  to  the  recurring  issues  of  America’s  early  years. 
Mark  Holbrook,  in  his  article  tying  acquisition  reform  initiatives  to  contract  cost  variance, 
attempts  to  ascertain  which  initiatives  have  had  the  most  impact  (2003).  Holbrook  demonstrates 
a  few  weak  impact  correlations,  but  more  important  is  that,  as  seen  in  the  previous  section,  the 
cycle  of  reform  and  analysis  has  been  ongoing  since  the  1770’s.  Similarly,  Christensen 
attempted  to  analyze  cost  overruns  in  defense  contracting  (1993).  His  research  focused  less  on 
the  cause  than  the  inevitability  that  a  contract  more  than  fifteen  percent  complete  will  never 
recover  from  a  cost  overrun.  Just  like  the  frigates  and  muskets  mentioned  earlier  as  well  as  other 
examples  cited  by  both  Nagle  (1992)  and  Jones  (1999),  cost  overruns  have  plagued  the  system 
since  the  early  1800s. 

Other  examples  of  repeating  history  include  Lowering  the  Cost  of  Federal  System 
Acquisition  (Kasser,  1996),  Spare  Parts  Horror  Stories  (“Spare  Parts”,  1997),  and  Back  to 
Basics  (Dornheim,  2006).  Kasser  recommends  moving  system  testing  and  inspection  to  an 
earlier  phase  of  the  development  cycle  in  order  to  catch  and  repair  anomalies  before  they  impact 
the  cost  of  the  acquisition  process.  This  is  a  simple  adaptation  of  the  theory  of  constraints  idea 
of  putting  the  inspectors  before  the  bottleneck  (Goldratt,  1984).  Arguably  this  is  a  fine  idea,  but 
history  demonstrates  that  inserting  and  extracting  inspectors  from  the  process  has  little  long  tenn 
effect  (Nagle,  1992).  Spare  Parts  Horror  Stories  chronicles  yet  another  instance  of  procurement 
problems  halting  the  mission.  The  article  notes  that  the  government  waits  longer  and  pays  more 
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for  Unmanned  Aerial  Vehicle  parts  from  the  contractor  than  from  procuring  the  parts  at  a  local 
engineering  supply  company.  Again,  this  brings  back  the  earlier  arguments  against  profiteering. 
However,  some  may  argue  that  the  UAV  technology  justifies  the  premium  on  the  common  use 
parts  (Nagle,  1992;  Jones,  1999).  Dorheim  chronicles  the  Air  Force’s  failed  attempt  to  contract 
out  all  facets  of  the  Evolved  Expendable  Launch  Vehicle  (EELV)  Program  (including 
management)  (2006).  After  numerous  setbacks,  the  program  was  returned  to  a  cost-plus  contract 
with  full  government  technical  oversights.  This  method  has  systemic  historical  problems, 
attributable  to  either  profiteering  or  the  government  being  the  sole  user  (Nagle,  1992;  Jones, 
1999).  The  reason  cited  in  the  EELV  case  is  the  failure  of  a  substantial  commercial  market  to 
materialize.  In  other  words,  since  the  government  is  the  sole  user  and  more  importantly  sole 
funder  of  the  technology,  government  oversight  is  required  to  ensure  its  interests  are  preeminent. 

Even  relatively  “new”  ideas  such  as  spiral  acquisition  and  cost  as  an  independent  variable 
(CAIV)  are  reflected  in  US  acquisition  history.  Wayne  Johnson  argues  that  spiral  acquisition  has 
positive  and  negative  implications.  Among  the  positives  are  the  ability  to  insert  new  technology, 
speed  to  the  war  fighter,  continuous  improvement,  and  knowledge  retention.  The  drawbacks 
include  an  eighty  percent  initial  solution,  possible  budget  cuts,  and  multiple  configurations  in  the 
field  (Johnson,  2002).  In  a  way,  the  1800  and  early  1900  government  letting  of  small  contracts 
to  keep  industry  proficient  while  continuing  internal  development  is  a  precursor  to  this  concept. 
This  strategy  has  proven  slow  in  peacetime  (reflecting  years  for  weapons  advancement  -  i.e.  the 
17  years  for  the  Ml  rifle),  mostly  due  to  congressional  funding  (Jones,  1999).  Johnson  even 
makes  several  arguments  on  the  possibility  that  congressional  and  government  financial  interests 
could  ruin  the  whole  spiral  development  concept  just  as  is  seen  in  past  attempts  (2002).  In 
Benjamin  Rush’s  article  on  CAIV,  a  strong  argument  is  made  for  including  cost  benefit  tradeoffs 
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in  acquisition  decisions  (1997).  While  this  sounds  like  a  new  and  great  idea,  the  concept  has 
existed  in  US  acquisition  since  1700.  At  that  time,  because  the  government  was  perpetually  low 
on  funds,  everything  involved  a  cost  tradeoff.  A  particular  similarity  to  today’s  concept  is 
buying  muskets.  The  level  of  technology  was  chosen  based  on  need  and  cost  (Nagle,  1992). 

This  has  been  carried  through  the  years  to  Truman’s  motto  to  get  “more  bang  for  the  buck”  in  the 
late  1940s  (Jones,  1999).  Although  Spiral  Acquisition  and  CAIV  are  now  formal  concepts,  the 
basic  premise  of  development  without  production  and  cost  as  a  variable  have  been  around  in  the 
US  acquisition  process  since  the  1800s. 

Summary 


Figure  6  The  Industry  Double  Helix 

This  section  provided  extensive  support  to  prove  that  no  one  acquisition  method  has 
proven  itself  better  than  others.  The  section  presented  an  in-depth  look  at  historical  success  and 
failures  of  the  acquisition  process  and  methods.  It  demonstrates  that  all  modem  acquisition 


24 


methods  have  been  attempted  throughout  the  US’s  history  and  validates  that  no  one  method  has 
proven  best.  In  fact,  history  is  ripe  with  successes  and  failures  of  each  method.  The  process  in 
use  today  is  simply  a  more  formalized  system  with  no  fundamentally  new  ideas  or  new 
acquisition  vehicles.  The  process  simply  follows  a  continuous  loop  of  integrating  and 
outsourcing  via  various  methods  as  illustrated  in  industry  by  Charles  Fine  in  Figure  6  (1998).  In 
order  to  increase  the  chance  of  developing  a  successful  system,  the  DoD  should  keep  options 
open  by  examining  all  reasonably  available  methods. 

Assumption  4:  External  development  is  the  method  of  choice  for  decision  makers 

Empirical  evidence  suggests  this  is  true.  Most  DoD  product  development  is  conducted 
and  completed  externally.  This  indicates  a  propensity  to  choose  external  over  internal 
development. 

Several  theories  exist  concerning  the  rationale  for  the  extreme  reliance  on  contractors  in 
the  acquisition  process.  One  is  that  the  government  does  not  have  the  talent  or  resources  to 
develop  or  transition  weapon  systems  (White  and  Deutch,  2004).  While  this  theory  may  be  true 
for  large  complex  weapons  systems  like  the  F-22,  this  thesis  contends  that  it  is  not  true  for  less 
complex,  mostly  off-the-shelf  technology  (i.e.  the  UAV  previously  mentioned).  Government 
entities  have  designed  working  weapons  systems  and  a  specific  recent  accomplishment  will  be 
covered  in  the  next  section  (Godbolt  and  Hawk,  2005). 

Others  argue  that  the  government  has  long  used  the  procurement  process  to  further  social 
and  economic  objectives  (Nagle,  1992:  1).  Guttman  makes  the  case  that  the  current  political 
climate  favors  a  small  government,  but  no  individual  or  organization  wants  to  give  up  current 
government  benefits  provided  specifically  to  them.  Therefore,  the  only  way  to  reduce 
government  employees,  and  thereby  government  size,  without  a  change  in  benefits  is  to  privatize 
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and  contract.  This  method  does  reduce  the  number  of  people  on  the  government  payroll,  but  it 
does  not  necessarily  reduce  government  or  the  cost  of  government  (Guttman,  2003).  If  this 
argument  has  merit,  then  a  contracted  or  privatized  system,  even  if  more  expensive,  would  be 
preferable  to  government  development  regardless  of  cost.  However,  one  would  hope  this 
argument  does  not  extend  to  cost  of  life.  If  not,  then  speed  to  the  war  fighter  would  be  important 
and  still  be  a  valid  argument  for  internal  development  over  external  development. 

Another  argument  for  the  existence  of  a  large  contractor  presence  in  the  acquisition 
process  is  that  outsourcing  is  cheaper.  Numerous  articles  tout  the  benefits  of  outsourcing  and 
demonstrate  initial  savings,  but  very  few  give  any  quantifiable  evidence  that  outsourcing  is 
cheaper  in  the  long  term  (Poole,  1980;  Kotabe,  1989;  Bettis  and  others,  1992;  Quinn,  1992; 
D’Aveni  and  Ravenscraft,  1994).  In  fact  numerous  US  Government  Accountability  Office 
(GAO)  reports  point  to  weapons  system  cost  overruns,  lack  of  cost  savings,  and  general 
overpricing  of  goods,  such  as  a  $10  washer  sold  to  the  government  for  $127  (“Spare  Parts”, 
1997). 

In  short,  general  arguments  against  government  development  of  weapons  systems  do  not 
seem  to  have  merit  except  when  concerning  the  most  complex  and  expensive  systems.  However, 
other  arguments  detail  specific  problems  with  the  way  the  government  currently  acquires 
systems  that  do  have  merit,  but  can  be  remedied  through  a  simple  change  in  mindset.  For 
instance,  critics  point  to  the  fact  that  government  scientists  have  taken  on  a  managerial  role.  The 
true  technical  work  has  been  given  to  contractors;  government  personnel  have  been  relegated  to 
contract  managers  (Suddarth,  2004).  Suddarth  views  this  as  a  two  prong  problem.  He  addresses 
and  suggests  remedies  to  both  the  reduction  in  the  number  of  and  current  role  of  government 
engineers.  The  reduction  in  engineers  reduces  the  feasibility  in  building  strong  internal 


26 


capability,  while  the  current  usage  of  engineer  is  typically  to  monitor  contracts  (2004). 
Obviously,  if  government  scientists  are  no  longer  available  for  technical  work,  then  internal 
development  is  not  possible.  In  meetings  with  Air  Force  Research  Laboratory  (AFRL) 
personnel,  this  is  the  second  most  referenced  problem  in  undertaking  internal  development 
efforts  (interviews).  However,  personnel  levels  and  career  paths  are  issues  that  are  under 
governmental  control  and  can  be  rectified  (Suddarth,  2004). 

Critics  also  point  to  the  lack  of  funding  for  research  and  development.  Suddarth  notes 
that  most  adjustments  decrease  the  science  and  technology  budget  and  that  the  long  term  trend  to 
the  Science  and  Technology  budget  is  not  promising  (Suddarth,  2005).  Suddarth  points  out  two 
issues:  first,  that  the  development  budget  is  generally  shrinking,  and  second,  that  the  budget  is 
tightly  controlled  and  rigid,  leaving  no  room  for  spontaneous  internal  breakthrough  programs.  In 
other  words,  the  internally  developed,  technology  push,  working  directly  with  the  war  fighter 
program  may  not  be  possible  due  to  rigid  budget  and  bureaucratic  requirements.  In  fact,  during 
the  interviews  with  AFRL  personnel,  this  was  the  most  referenced  problem  with  undertaking 
internal  development  efforts.  Suddarth  presents  several  reasons  for  this  problem,  including 
pressures  from  Congress  to  reduce  duplication  of  effort  and  long-term  strategic  planning;  both  of 
these  issues  stifle  creativity  (2005).  Funding  constraints,  however,  are  another  issue  which  can 
be  remedied. 

Personnel  training  and  budgetary  issues  are  both  issues  that  have  merit  concerning  the 
internal  versus  external  development  debate,  however  the  core  of  this  thesis  is  to  compare  final 
results  of  internally  and  externally  developed  programs  rather  than  explore  remedies  to  the 
system’s  bureaucratic  issues.  It  is  possible  that  internal  development  could  be  faster,  cheaper, 
and  more  capable,  but  require  major  guideline  and  constraint  revisions  to  implement. 
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Concepts  Important  to  this  Research 


The  purpose  of  this  thesis  is  to  show  that  not  only  has  exclusively  utilizing  one 
development  method  proven  impossible  throughout  history,  but  it  also  continues  to  be 
problematic  today.  The  AFRL  Strategic  Plan  under  General  Paul  stated  that  80%  of  research 
efforts  should  be  outsourced;  however,  there  is  no  factual  or  empirical  rational  included.  Other 
government  documents,  such  as  OMB  Circular  A-76,  indicate  that  outsourcing  will  reduce  cost, 
and  that  same  idea,  discussed  below,  is  prevalent  throughout  industry.  As  with  other  industry 
fads  (i.e.  Total  Quality  Management,  Lean,  and  Six  Sigma),  the  DoD  and  Air  Force  undoubtedly 
assumed  all  claims  were  true  and  therefore  planned  to  save  money  through  outsourcing  efforts. 
The  concepts  used  in  this  research  draw  upon  all  the  concepts  included  in  the  theory,  goals,  and 
perils  surrounding  outsourcing  in  the  literature.  To  aid  in  explanation  and  later  in  formulating 
questions,  these  concepts  are  categorized  into  sub-areas  for  this  research.  These  areas  are  cost, 
feasibility,  capability,  training,  speed,  customer  relation,  concurrency,  and  guidance. 

Cost 

In  preliminary  questioning  of  subject  matter  experts,  cost  was  one  of  the  most  cited 
reasons  for  making  a  contracting  decision.  This  is  not  surprising  since  government  is  usually 
focused  on  cost.  Recent  articles  and  reports  touting  the  savings  involved  in  contracting  out  what 
was  once  organic  capability  have  been  growing  throughout  the  1980s  and  1990s  (Poole,  1980; 
Kotabe,  1989;  Bettis  and  others,  1992;  Quinn,  1992;  D’Aveni  and  Ravenscraft,  1994;  Lei  and 
Hitt,  1995;  Rasheed,  2000;  Grasso,  2003).  Many  believe  that,  especially  in  the  bureaucratic  case 
like  the  government,  cost  will  certainly  be  reduced  by  outsourcing  (D’Aveni  and  Ravenscraft, 
1994).  This  may  have  been  demonstrated  in  initial  competition  through  Most  Efficient 
Organization  (MEO)  efforts  in  which  many  base  functions  were  replaced  by  cheaper  contractor 
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efforts  throughout  the  1990s  (GAO,  2000).  However,  this  is  usually  a  one  time  effort  so  any 
potential  savings  might  be  offset  by  future  contract  cost  increase.  The  competition  method  for 
these  efforts  may  also  be  skewed.  For  example,  a  military  Airman’s  cost  per  hour  is  calculated 
based  on  all  future  earnings  to  include  retirement.  This  creates  a  cost  target  that  is  easy  to  beat. 
Since  many  Airmen  separate  prior  to  retirement,  this  cost  estimate  seems  skewed  toward 
outsourcing.  Other  examples  involve  not  including  the  price  of  the  cost  analysis  and  contracting 
process  in  the  total  cost  to  contract  an  effort  as  well  as  failure  to  include  ad-hoc  services  typically 
performed  by  government  employees  that  will  require  additional  pay  for  a  contractor.  One  final 
example  that  can  skew  the  cost  estimate  toward  the  government  is  not  including  tax  advantages 
in  hiring  private  providers  (Carver,  1989;  McEntee,  1985;  O’Looney,  1998).  Unfortunately, 
both  sides  of  the  argument  include  bias. 

In  all  this,  initial  cost  savings  is  considered  the  central  issue.  Long-term  profitability,  or 
in  the  case  of  government,  lowest  cost,  cannot  be  found  in  short  term-cost  savings  (Rasheed, 
2000;  Fine,  1998).  Some  form  of  return  on  investment  calculation  would  be  more  appropriate. 

Feasibility 

Another  central  theme  in  outsourcing  is  best  business  practice  or  strategic  outsourcing. 
This  could  be  described  as  concentrating  on  areas  of  proficiency  while  outsourcing  the  remaining 
areas  (Quinn,  1992).  DoD  laboratories  encompass  a  large  area  of  expertise.  Even  within  AFRL, 
areas  from  sensors,  to  software,  to  human  effectiveness,  to  propulsion  are  encountered.  While  a 
central  authority  could  direct  all  laboratories  to  concentrate  only  on  a  certain  area  while 
outsourcing  the  rest  (which,  in  essence,  forces  them  to  be  identical),  it  is  more  logical  to  the 
author  to  assume  that  they  are  not  all  alike.  Therefore,  rather  than  concentrating  on  the  area  in 
which  each  laboratory  excels;  this  thesis  concentrates  on  internal  feasibility.  In  other  words,  this 
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thesis  seeks  a  methodology  to  present  and  produce  ideas  internal  to  the  laboratory  rather  than 
assume  that  all  work  be  contracted. 

Capability 

A  third  area,  capability,  might,  at  first,  seem  similar  to  best  business  practice  or  the 
laboratory’s  area  of  expertise,  but  it  is  meant  to  gauge  what  Charles  Fine  in  Clock  Speed  refers 
to  as  a  companies  “core  competencies”  (1998).  Some  authors,  like  Fine,  think  it  is  possible  for 
industry  to  swing  too  far  into  outsourcing  while  pursuing  lower  costs.  They  loose  their  internal 
capability  and  competencies  and  become  completely  reliant  on  others  (Bettis  and  others,  1992; 
Fine,  1998).  In  the  business  world  this  can  mean  the  end  of  a  company,  but  in  a  laboratory 
environment  it  could  mean  the  inability  to  do  in-house  work. 

Training 

Some  authors  counter  that  training  is  enough  to  keep  personnel  sufficiently  technically 
proficient  to  maintain  their  core  competencies  while  enjoying  the  cost  benefits  of  outsourcing 
(Bettis  and  others,  1992).  This  leads  to  the  question  concerning  what  training  is  required  within 
DoD  laboratories  to  maintain  proficiency  and  competency.  Guidance,  at  least  within  AFRL, 
mandates  80  hours  of  training  every  two  years.  These  hours  can  be  satisfied  in  a  number  of  ways 
ranging  from  mandatory  Air  Force  recurring  training  to  doctoral  degree  classes. 

Some  authors  take  this  argument  one  step  further  stating  that  technical  proficiency  in  all 
areas  may  no  longer  be  required.  The  company  is  paying  the  contractor  for  that  proficiency  so 
personnel  only  need  to  be  good  contract  monitors.  In  fact,  some  feel  that  too  much  technical 
training  may  lead  to  undue  project  bias  (Sahlin,  1998;  O’Looney,  1998).  In  this  argument,  it 
would  not  be  an  issue  for  a  DoD  laboratory  to  loose  the  ability  to  do  in-house  work  or  to  loose 
technical  expertise. 
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Speed 

Another  issue  often  brought  up  in  relation  to  outsourcing  is  speed.  In  the  case  of  DoD 
developments,  this  involves  the  time  required  to  get  a  product  to  the  war  fighter.  Usually,  this 
topic  is  related  to  best  business  practice  such  that  if  everyone  concentrates  on  their  core 
competencies,  then  products  can  be  brought  to  market  faster  (Bettis  and  others,  1992;  Kessler 
and  others,  2000).  This  means  that,  in  theory,  contracting  efforts  actually  decrease  product 
development  time.  However,  other  authors  disagree  with  this  premise  and  provide  data  to 
dispute  it  (Kessler  and  others,  2000).  The  rationale  is  that  streamlining  the  effort  internally  and 
avoiding  the  contracting  process  can  speed  production. 

Customer  Relations 

Customer  Relationship  is  another  area  sometimes  cited  in  reference  to  outsourcing. 

Some  arguments  claim  that  outsourcing  puts  a  strain  on  customer  relations  since  customers  must 
go  through  an  intermediary  to  reach  the  product  developers.  Others  argue  that  outsourcing 
allows  a  company  to  focus  on  customer  needs  rather  than  product  development  (Rasheed,  2000; 
Hagel  and  Singer,  1999).  In  industry,  the  later  premise  leads  some  firms  to  concentrate  on 
product  development  while  outsourcing  customer  relations.  In  DoD,  the  process  typically  occurs 
through  contracting  design  and  build  while  a  government  employed  program  manager  continues 
customer  relations. 

Concurrency 

Although  initial  MEO  efforts  have  always  included  the  initial  government  team  (those 
currently  doing  the  work),  future  re-competition  efforts  do  not.  Similarly,  competitive  bids  for 
laboratory  contracts  are  usually  between  contractors  only.  However,  some  recent  cases  have 
shown  better  long  tenn  results  when  public  sector  providers  are  included  in  the  mix  of  potential 
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competitors  whenever  possible  (Flanagan  and  Perkins,  1995;  O’Looney,  1998).  This  increases 
competitors  and  can  guard  against  low-balling  and  price  escalation.  In  the  laboratory 
environment,  this  means  at  least  maintaining  the  capability  for  an  internal  development  effort  so 
that  these  benefits  remain  possible.  The  actual  process  of  internally  competing  has  been  dubbed 
concurrency  for  the  purpose  of  this  thesis  and  the  surveys  contained  therein. 

Guidance 

As  stated  earlier,  a  laboratory  can  be  forced  through  direction  to  always  choose  a  certain 
path.  While  this  area  is  not  a  part  of  the  reviewed  literature  dealing  with  outsourcing,  company 
guidance  can  play  a  very  important  role  in  the  direction  of  the  company  (Hall,  1984;  Peters, 
1987).  This  area  will  be  very  important  if  the  employees  understand  and  adhere  to  any 
requirements  imposed  by  higher  level  agencies.  This  area  may  also  be  problematic  if  employees 
completely  ignore  any  guidance. 

Summary 

The  literature  surrounding  or  supporting  each  assumption  was  briefly  reviewed  to  either 
prepare  the  assumption  for  testing  or  prove  it  valid.  The  concepts  important  to  outsourcing  in  the 
literature  were  reviewed  one  by  one  and  compared  to  historical  occurrences. 

The  goal  of  this  thesis  is  not  to  prove  that  government  development  is  the  optimal 
solution,  but  rather  to  investigate  if  one  method  is  actually  superior  to  the  other  if  they  are  both 
equal.  In  other  words,  to  ensure  the  greatest  chance  of  product  development  success  by  showing, 
as  Jefferson  Davis  said,  that  “both  are  needed”  (Nagle,  1992:  89). 
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ITT.  Methodology 


Introduction 

This  chapter  covers  the  methodology  chosen  to  conduct  this  research  and  the  tools  and 
techniques  used  for  data  collection.  Although  extensive  literature  exists  surrounding  the  topic, 
there  is  no  official  Air  Force  guidance  regarding  when  to  develop  internally  versus  externally. 
Additionally,  a  few  recent  publications  have  argued  the  wisdom  of  contracting  out  to  save  cost 
(Fine,  1998).  This  coupled  with  the  reported  propensity  for  DoD  laboratories  to  choose  external 
development  led  to  the  thesis  purpose.  Data  availability  as  well  as  time  and  resource  limitations 
led  to  the  determination  that  an  exploratory  case  study  was  the  best  approach. 

Qualitative  Research  Method 

The  design  chosen  is  an  exploratory  case  study  approach.  This  does  not  detract  from  the 
relevancy  or  contributions  of  this  thesis.  A  common  misconception  is  that  various  research 
strategies  are  hierarchically  arranged,  when  in  fact  they  are  all  equal  (Yin,  2003).  Depending  on 
the  type  of  research  question,  extent  of  researcher  control,  and  amount  of  contemporary  versus 
historical  focus,  one  method  may  be  more  applicable  than  another.  In  this  case,  the  objective  of 
comparing  the  results  of  internal  laboratory  development,  testing,  and  fielding  of  small  weapons 
systems  was  accomplished  though  the  capture  of  the  everyday  circumstances  and  conditions  that 
were  present  (Yin,  2003).  The  recognized  experts  in  qualitative  design  are  Robert  Yin  and  John 
Creswell  (Espy,  2006;  Suarez,  2006),  and,  as  a  result,  their  methods  are  relied  upon  heavily  in 
this  case  study  design  and  analysis.  Both  agree  that  a  case  study  is  ideally  suited  for  the  study  of 
a  real  life  phenomenon  without  clearly  defined  boundaries  (Yin,  2003;  Creswell,  2003).  This 
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methodology  essentially  describes  the  situation  of  internal  weapons  development  presented  in 
this  study. 

The  Exploratory  Case  Study 

Yin  defines  a  case  study  as  a  historical  account  with  the  added  benefit  of  direct 
observation  and  interviews  (2003).  As  applied  to  this  thesis,  an  exploratory  case  study  is  most 
applicable;  the  value  of  the  knowledge  and  exact  questions  requiring  answers  are  not  yet  known 
(Creswell,  1994).  Although  great  effort  has  been  made  to  base  the  questions  and  assumptions  in 
grounded  theory,  results  of  the  interviews  could  prove  insightful  in  contributing  to  or  modifying 
the  interview  questions  for  a  larger  study.  In  order  to  perform  the  research,  study  questions  must 
be  developed,  a  unit  of  analysis  defined,  a  method  of  gathering  data  chosen,  and  a  criteria  for 
interpreting  data  selected  (Yin,  2003).  Yin  defines  this  as  the  study  blueprint. 

The  overall  research  question  for  this  study  is:  Is  there  a  quantifiable  difference  in  war 
fighter  capability  and  delivery  between  internally  and  externally  developed  small  weapons 
systems  and,  if  so,  why?  The  decision  to  internally  develop  a  system  or  contract  it  out  has  a  long 
history.  The  decision  often  is  not  made  by  one  person  or  even  by  one  department.  Current  DoD 
guidance  on  whether  to  develop  internally  or  externally  is  near  non-existent  beyond  basic 
contract  cost  comparison  requirements.  These  realizations  led  to  several  investigative  questions: 

-  Which  involves  less  cost? 

•  Will  concurrent  efforts  reduce  cost? 

-  Which  produces  better  capability? 

•  Can  internal  be  done? 

•  Does  one  produce  a  better  customer  interface? 

•  What  defines  personnel  capability? 


34 


•  Does  one  produce  better  personnel  capability? 

-  Which  provides  faster  delivery? 

•  Are  there  process  problems  in  either  acquisition  method 

-  Are  personnel  given  any  guidance  on  the  issue? 

Since  an  exploratory  or  pilot  study  typically  consists  of  a  sample  of  data  (in  the  case  of 
this  thesis  those  will  be  individual  interviews),  the  unit  of  analysis  is  the  individual  being 
questioned  (Creswell,  2003).  The  analysis  will  be  summarized  at  the  AFRL  level. 

Although  many  methods  of  data  collection  for  a  case  study  exist,  an  interview  process  is 
most  applicable  for  an  exploratory  situation.  Interviews  reveal  more  than  history  and  are  more 
fluid  than  surveys.  Interviews  also  provide  more  information  to  a  researcher  because  the 
researcher  can  ask  insightful  follow-up  questions  to  any  initial  questions  on  the  survey 
instrument.  Analysis  proves  more  difficult  since  varied  verbal  responses  must  be  quantified  into 
units  usable  in  research  comparisons. 

In  this  case,  the  criteria  for  interpreting  the  data  follow  directly  from  the  method  of  data 
collection.  Since  an  interview  process  is  used,  the  only  option  is  qualitative  data  analysis.  For 
qualitative  data  analysis,  three  methods  are  most  applicable.  These  are  pattern  matching,  in 
which  several  pieces  of  information  across  the  interviews  can  be  related;  theme  development,  in 
which  recurring  themes  are  magnified  through  repetition;  and  explanation  building,  in  which 
each  interviewee  connects  a  piece  of  the  puzzle  (Yin,  2003;  Campbell,  1975;  Trochim,  1989). 

Data  Gathering 

The  interview  process  for  this  exploratory  study  was  limited  to  employees  within  AFRL. 
The  time  consuming  nature  of  an  interview  process  and  Creswell’s  optimal  number  of  20  to  30 
per  pilot  study  required  additional  restrictions  on  the  scope  of  the  process  (2003).  By  having  all 
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units  of  analysis  in  AFRL,  the  data  could  be  more  concentrated,  the  timetable  more  compressed, 
and  the  results  more  applicable  and  transferable  to  a  particular  unit. 

After  selecting  AFRL  as  the  primary  test  case,  a  Subject  Matter  Expert  (SME)  within 
AFRL  provided  a  list  of  potential  interviewees  ensuring  a  cross  section  of  directorate,  age, 
position,  and  years  of  service.  The  only  requirement  for  inclusion  in  the  interview  process  was  at 
least  three  years  in  a  government  research  related  position.  This  requirement  was  included  to 
ensure  units  of  analysis  were  qualified  and  knowledgeable  enough  to  provide  responses  to  the 
interview  questions.  The  proposed  questions  were  also  reviewed  and  tested  by  a  small  group  of 
SMEs  to  ensure  accurate  and  non-repetitive  questions  as  well  as  proper  word  selection.  The 
questions  were  left  open-ended  to  allow  further  elaboration  (Creswell,  1994). 

The  perspective  interviewees  were  then  contacted  by  phone  and  e-mail  to  schedule  face- 
to-face  individual  interviews.  Usually  these  interviews  were  conducted  within  the  office  of  the 
interviewee,  but,  on  occasion,  were  accomplished  in  a  substitute  setting  (such  as  a  conference 
room).  A  few  interviews  had  to  be  conducted  by  phone  due  to  the  geographic  separation  of  some 
AFRL  directorates  from  Wright-Patterson  AFB.  Each  interviewee  was  advised  of  the  purpose, 
sponsors,  and  disposition  of  the  research  and  assured  that  participants  would  not  be  personally 
identifiable.  Only  the  researcher  was  aware  of  the  actual  names  of  interviewees  and  only  the 
thesis  committee  had  access  to  the  interview  raw  data  to  ensure  privacy.  They  were  also 
presented  a  privacy  act  statement  as  required  by  the  AFRL  Internal  Review  Board  which  granted 
pennission  to  conduct  the  interviews. 

The  interview,  itself,  was  a  focused  interview  (Merton  and  others,  1990)  revolving 
around  the  basic  areas  presented  in  chapter  2  and  some  additional  general  question  areas.  This  is 
summarized  in  Table  4.  Notes  were  taken  during  each  interview  and  recordings  were  made 
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when  allowed  by  the  interviewee.  Before  beginning  the  main  line  of  questioning,  each 
interviewee  was  advised  of  the  study’s  scope  and  the  definitions  of  in-house,  external, 
concurrent,  and  system  as  they  applied  to  the  line  of  questioning.  The  focus  areas  and  initial 
interview  questions  asked  are  presented  in  Table  4.  Follow-up  questions  were  asked  as  needed 
to  completely  understand  interviewee  responses;  additional  questions  concerning  funding 
streams  and  transition  were  only  asked  to  individuals  who  commented  upon  issues  with 
transition. 

Table  4  Interview  Instrument 


AREA 


QUESTIONS 


Feasibility  Are  you  aware  of  any  avenue  to  present  and  produce  internally  developed 

ideas? 

Is  it  possible  for  AFRL  to  create  a  working  prototype  of  a  system? 

What  would  AFRL  need  to  accomplish  this? 

What  stumbling  blocks  currently  stand  in  your  way? 

Have  you  been  involved  in  an  internal  development  effort? 

Concurrency  Are  you  encouraged  to  promote  competition? 

Is  this  ever  done  though  direct  competition  via  multiple  efforts? 

Could  laboratory  development  provide  direct  competition? 

Have  you  been  involved  in  a  concurrent  effort? 


Customer  Relation  How  often  must  you  translate  customer  requests  to  industry? 

Do  your  industry  contractors  interface  directly  with  the  customers? 
Would  it  be  easier  to  implement  customer  needs  if  the  project  were 
internal? 


Capability  Are  your  government  employed  personnel  technically  proficient? 

Do  they  spend  their  time  developing  systems  or  providing  oversight  to 
industry  developed  systems? 

Would  they  be  more  technically  proficient  if  directly  involved  in  system 
development? 
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Continuation  of  Table  4  Interview  Instrument 


Training  How  do  you  ensure  your  engineers  remain  technically  proficient? 

Is  training  centered  on  engineering  or  contract  management? 

In  your  opinion,  what  can  be  done  to  improve  the  situation? 

Do  government  personnel  really  need  engineering  experience  in  today's 
environment? 

Cost  Would  it  cost  more  to  develop  internally? 

How  would  concurrent  development  apply  to  or  affect  your  area? 

If  your  product  was  internally  developed,  would  it  be  cheaper  to  produce 
or  sustain? 

Speed  to  War  Fighter  Could  an  internally  developed  system  be  fielded  faster  than  an  externally 

developed  one? 

Can  the  government  perform  tests  that  are  required  for  fielding  faster 
than  industry? 

What  roadblocks  do  internally  developed  systems  or  industry  developed 
systems  have  an  advantage  in  surmounting  to  reach  IOC? 

What  guidance  do  you  follow  when  deciding  what  to  develop  internally 
and  what  to  contract  to  industry? 

Do  you  have  any  unwritten  rules  on  the  subject? 

In  your  opinion,  what  would  be  the  single  most  important  issue  in  a 
debate  between  internally  developing  and  externally  developing 
systems? 


Guidance 


General 


Data  Analysis 

After  the  data  collection,  analysis  was  conducted  involving  techniques  for  case  studies 
recommended  by  Creswell  and  Yin  (1994,  2003,  2003)  beginning  with  Creswell’s  spiral  which 
prescribes  organization,  perusal,  classification  and  synthesis  as  depicted  in  Figure  6.  The  data  is 
broken  up  into  phrases  which  can  be  stored  in  a  database.  It  is  then  reviewed  repeatedly  to 
understand  overarching  concepts  and  meanings  so  that  the  data  could  be  classified  into  common 
categories  and  summarized.  Finally,  it  is  used  to  form  propositions  and  hypotheses  concerning 
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relationships.  This  combined  with  the  pattern  matching,  theme  development  and  explanation 
building  mentioned  earlier  provided  results  for  this  exploratory  study  and  possibilities  for  future 
study  (Trochim,  1989;  Yin,  2003). 


Reliability  and  Validity 


Several  preventative  measures  were  incorporated  to  ensure  validity  and  reliability  in  the 


study.  First,  the  presence  and  input  of  SMEs  during  the  overall  process  and  third  party  review  of 
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data  ensured  construct  validity  and  provided  internal  feedback  (Yin,  2003).  Second,  to  validate 
data  synthesis,  an  external  audit  by  colleagues  not  associated  with  the  study  was  performed  to 
determine  if  they  reached  conclusions  similar  to  the  researcher.  Third,  to  prevent  any  inadvertent 
researcher  bias,  participants  were  randomly  selected.  Finally,  the  case  study  skills  and 
procedures  used  were  based  on  texts  and  books  by  experts  in  research  which  are  well 
documented  to  ensure  reliability  (Yin,  2003,  Creswell,  1994). 

Overall,  the  researcher  attempted  to  gain  an  understanding  of  what  could  feasibly  be  done 
to  modify  or  change  current  practices  and  the  rationale  or  perceptions  that  guided  the 
malleability  of  the  status  quo.  The  researcher  remained  adaptive  and  flexible  during  questioning 
to  ensure  opportunities  for  new  insight  were  not  overlooked  and  refrained  from  any  personal  bias 
from  past  experience.  This  analysis  allowed  the  researcher  to  better  understand  internal  and 
external  capabilities  of  DoD  laboratories  in  general  and  AFRL  in  particular. 
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IV.  Analysis 


Introduction 

The  focus  of  this  effort  is  to  evaluate  the  extent  to  which  DoD  laboratories  are  capable  of 
implementing  and  delivering  internally  developed  small  weapons  systems  versus  those  externally 
developed.  The  review  of  the  literature  became  the  baseline  for  the  interview  instrument 
questions.  Interviews  were  then  conducted  with  a  wide  range  of  laboratory  personnel.  Certain 
initial  responses  required  follow-up  questions  to  further  explore  each  topic  to  get  a  full 
understanding  of  the  interviewee’s  views  on  the  subject.  The  typical  interview  consisted  of 
approximately  70  minutes  of  discussion.  Only  two  were  less  than  45  minutes  and  one  continued 
for  over  120  minutes.  Based  on  Yin  and  Creswell  exploratory  case  study  methodology,  the  goal 
for  completed  interviews  was  between  twenty  to  thirty.  For  this  study,  twenty-nine  subjects  were 
interviewed  (not  including  the  subject  matter  expert  who  was  discussed  in  chapter  three). 

This  chapter  discusses  the  data  captured  from  those  twenty-nine  subjects  and  patterns  and 
themes  that  were  uncovered  in  the  subsequent  analysis.  The  following  sections  will  cover 
analysis  of  the  data,  note  any  themes  that  occurred  for  each  question,  and  include  some 
individual  comments  that  were  significant. 

Interview  Question  Analysis 

In  addition  to  the  questions  within  each  interview  focus  area,  each  individual  was  asked 
for  information  on  distinguishing  factors  including  where  they  worked,  how  many  total  years  of 
DoD  laboratory  experience  they  had  completed,  types  of  money  with  which  they  dealt  (61,  62, 
etc),  what  TRL  range  they  managed  (if  any),  and  whether  they  were  on  a  management  or 
scientific  track.  This  infonnation  was  used  for  several  purposes.  First,  it  was  used  to  ensure  an 
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adequate  cross  section  of  laboratory  personnel  was  included  in  the  sample.  The  sample  includes 
a  wide  range  of  all  distinguishing  factors,  including  representatives  from  all  but  one  of  the  ten 
AFRL  directorates.  Second,  any  correlations  between  these  statistics  and  the  answers  given  to 
the  questions  might  indicate  influence  on  that  response  from  another  source.  For  instance,  if 
everyone  in  one  directorate  provides  similar  responses,  but  their  responses  are  different  from 
other  directorates,  this  could  indicate  something  evident  in  only  that  directorate’s  culture  or 
directives.  Lastly,  the  absence  of  any  evident  correlations  to  specific  categories  enables  easier 
generalization  of  individual  responses  to  the  overall  group  rather  than  a  specific  portion  of  the 
group. 

When  it  was  evident  that  responses  were  similar  based  on  a  distinguishing  factor,  relevant 
information  was  noted  in  that  questions  analysis.  For  some  questions,  general  comments  by 
individuals  are  included  as  additional  information.  In  general,  specific  comments  are  attributable 
to  a  single  individual.  Where  two  or  more  individuals  presented  similar  ideas,  this  is  noted  in  the 
analysis. 

Area  One:  Feasibility 

Ql-1)  Are  you  aware  of  any  avenue  to  present  and  produce  internally  developed  ideas? 
The  goal  of  this  question  was  to  delve  into  the  first  connection  in  the  push  question.  Was  there  a 
way  for  personnel  to  push  internal  ideas  into  the  funding  stream  at  least  for  initial  research? 
Without  a  way  to  push  an  idea  to  funding,  the  likelihood  of  any  internal  project  is  unlikely. 

Some  follow-up  questions  included:  Is  the  funding  available  immediately  or  must  you  wait  on  the 
two  year  PPBS  cycle,  and  Is  this  specific  to  your  directorate? 
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Almost  everyone  had  access  to  funding  for  their  internally  developed  ideas.  However, 
only  about  half  of  those  could  get  access  to  money  for  a  good  idea  without  waiting  for  at  least  a 
two  year  PPBS  cycle.  The  overall  results  are  presented  in  Table  5. 


Table  5  Push  Metrics 


Themes 

#  of  responses 

%  of  participants 

#  of  participants 

Yes;  There  is  a  process  to  get  money 
immediately  for  good  ideas. 

12 

41% 

29 

Yes;  There  is  a  process  to  get  money  for 
good  ideas  by  waiting  2  years  on  the  PPBS 
cycle. 

12 

41% 

29 

No;  There  is  no  process  to  get  a  good  idea 
funded. 

5 

17% 

29 

The  process  most  referenced  gaining  access  to  money  was  briefing  the  individual’s  chain 
of  command  on  their  idea.  This  was  mentioned  by  twenty  of  the  twenty-nine  subjects.  Of  those, 
nine  described  a  formal  recurring,  quarterly,  or  semi-annual  process  to  present  research  ideas  for 
funding  and  the  others  described  a  more  ac-hoc  process.  Six  of  the  twenty-nine  cited  some  form 
of  management  reserve  to  allow  immediate  funding  of  projects  within  their  directorate. 

Programs  specifically  mentioned  were  the  innovation  research  fund,  the  innovation  idea  fund, 
and  the  high  impact  technology  program.  Other  processes  described  for  immediate  internal  idea 
funding  were  moving  funds  from  a  lower  priority  project  or  reallocating  unspent  quarterly  funds 
from  other  projects. 

Six  of  the  twenty-two  cited  bringing  in  external  funds  as  a  way  to  immediately  fund  an 
internal  idea.  Although  most  of  those  noted  any  idea  that  brought  in  extra  funds  was  worth 
research,  one  of  the  individuals  interviewed  claimed  that  this  only  worked  when  researching 
something  sanctioned  by  their  directorate. 
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The  nature  of  respondent  answers  were  generally  tied  to  their  directorate.  All  PR 
respondents  noted  that  external  funding  was  always  welcome.  SN  and  VA  respondents 
repeatedly  cited  fonnal  processes  to  fund  ideas.  However,  at  least  one  individual  in  both  SN  and 
VA  was  not  aware  of  their  directorate’s  push  options. 

Ql-2)  Is  it  possible  for  AFRL  to  create  a  working  prototype  of  a  system? 

The  goal  of  this  question  was  to  get  the  interviewee’s  basic  opinion  of  AFRL’s  overall  internal 
capability.  A  typical  follow-up  question  included:  Would  this  system  be  on  par  with  a  contractor 
developed  system?  Twenty-eight  of  twenty-nine  respondents  thought  AFRL  could  create  a  good 
prototype.  Of  these,  twenty-four  individuals  said  this  could  always  be  done,  two  said  it  could  be 
done  for  software  but  not  hardware,  one  said  portions  of  systems  could  be  done,  and  one  said 
prototyping  through  simulation  was  possible  and  adequate  for  a  basic  prototype.  One  individual 
from  SN  went  further  to  describe  standing  up  a  laboratory  rapid  product  development  center  in 
collaboration  with  AFIT  to  design  and  fabricate  internal  ideas. 

The  one  individual  who  thought  AFRL  could  not  create  a  good  prototype  commented  that 
AFRL  is  only  an  integrator,  not  a  developer  of  technology.  This  response  was  specific  to  this 
individual  as  others  from  the  individual’s  directorate  thought  AFRL  could  create  a  quality 
prototype. 

Ql-3)  What  would  AFRL  need  to  accomplish  this? 

This  question  was  a  standard  follow-up  to  Ql-2.  If  Ql-2  was  a  yes,  this  question  was  typically 
re-phrased:  Could  you  accomplish  this  with  what  you  currently  have  available?  About  half  the 
respondents  thought  they  had  everything  required  to  create  a  prototype.  Those  who  responded 
no  generally  lacked  equipment,  facilities,  people,  skills,  or  funding.  Table  6  summarizes  the 
results. 
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Table  6  Internal  Prototype  Feasibility 


Themes 

#  of  responses 

%  of  participants 

#  of  participants 

We  have  everything  we  need. 

15 

52% 

29 

We  need  additional  equipment  and/or 
facilities. 

8 

28% 

29 

We  need  additional  people/skills. 

6 

21% 

29 

We  need  additional  funding 

5 

17% 

29 

Five  of  twenty-nine  cited  fabrication  or  machining  as  the  primary  skill  lacking.  Two 
cited  a  lack  of  software  or  modeling  skills.  One  individual  thought  systems  engineering  and 
transition  skills  were  needed. 

In  equipment,  three  of  twenty-nine  thought  that  the  government  lacked  machine  and 
fabrication  shops.  Two  others  indicated  that  better  access  to  wind  tunnels  was  required.  All  five 
of  these  individuals  were  in  agreement  that  the  facilities  were  once  available,  but  had  been 
downsized,  cutback  our  outsourced.  One  interviewee  discussed  the  early  to  mid-1990s  move 
toward  divesting  capability  considered  non-core  competencies  and  its  impact  on  the  lack  of 
fabrication  capability. 

Within  the  six  of  twenty-nine  respondents  that  required  additional  people  with  skills,  one 
respondent  also  added  that  one  government  employee  was  often  handling  five  to  eight  different 
projects. 

The  responses  to  this  question  seemed  tied  to  directorates.  Except  for  two  funding 
requests,  PR,  HE,  and  IF  personnel  generally  replied  they  needed  nothing  additional.  All  VA 
and  DE  interviewees  indicated  that  additional  resources  were  required. 

Ql-4)  What  stumbling  blocks  currently  stand  in  your  way? 
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This  question  was  meant  to  explore  deeper  issues  than  Ql-3.  It  typically  needed  clarification 
with  the  phrase:  This  can  include  personnel,  skills,  equipment,  political,  and  any  other  issues. 

Not  just  physical,  but  environment  and  culture.  Almost  seventy  percent  of  personnel  interviewed 
noted  political  issues  which  caused  stumbling  blocks  to  internal  development.  This  was,  by  far, 
the  largest  cause  of  road  blocks.  Table  7  summarizes  the  general  themes. 


Table  7  Stumbling  Blocks 


Themes 

#  of  responses 

%  of  participants 

#  of  participants 

Political 

20 

69% 

29 

Budget 

10 

34% 

29 

Skills/personnel./time 

4 

14% 

29 

Equipment 

2 

7% 

29 

For  political  road-blocks  two  main  themes  existed.  Ten  of  the  twenty  respondents 
discussed  their  directorate’s  paradigm  that  the  laboratory  exists  to  do  basic  research,  not  to 
develop  products  and  that  a  large  majority  of  research  should  be  done  through  contracts.  When 
asked  why,  the  most  often  cited  reason  was  personal  choice  of  the  directorate  chief.  The  other 
main  theme  was  teaming.  Six  of  the  twenty  responded  that  a  good  internal  effort  took  teaming 
across  directorates  and  internal  politics  made  that  difficult  even  when  the  people  with  the 
required  expertise  were  known.  When  asked  why  teaming  efforts  were  difficult,  squabbles  over 
budget,  manpower,  and  credit  for  results  were  cited  as  reasons.  One  individual  commented  that 
it  was  easier  to  move  money  from  the  Army  to  his  directorate  for  research  than  it  was  to  move 
money  from  one  directorate  to  another. 
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For  budget,  the  central  theme  was  lack  of  funding  in  general.  However,  three  individuals 
commented  on  congressional  earmarks.  The  comments  for  this  question  seemed  to  be  very 
specific  to  the  individual  and  their  experience  rather  than  any  particular  directorate  or  track. 

Ql-5)  Have  you  been  involved  in  an  internal  development  effort? 

This  question  was  not  the  main  emphasis  for  this  area,  but  a  leading  question  for  the  follow  up 
questions.  If  the  interviewee  answered  yes,  the  following  questions  were  asked:  Can  you  tell  me 
about  the  effort  and  product?  and  How  did  it  turn  out?  The  real  goal  of  this  question  was  to 
compare  the  results  of  internal  and  external  projects.  Twenty-five  of  twenty-nine  participants 
had  experience  with  an  internal  development  effort.  Six  of  those  twenty-six  drew  upon 
experience  from  the  1980s.  Almost  all  of  the  comments  about  the  programs  these  individuals 
were  involved  with  as  well  as  the  results  from  these  programs  were  positive.  No  one  thought  a 
contractor  could  have  done  a  better  job  for  a  cheaper  price.  Two  individuals  thought  their 
government  effort  should  have  included  a  better  transition/production  plan. 

The  comments  for  this  question  seemed  to  be  very  specific  to  the  individual  and  their 
particular  experience  rather  than  any  particular  directorate  or  track.  Not  even  years  of  laboratory 
experience  seemed  to  impact  whether  an  individual  had  been  involved  in  an  internal  effort. 

Area  Two:  Concurrency 

Q2-1)  Are  you  encouraged  to  promote  competition? 

Like  the  previous  area,  the  first  question  was  meant  to  gauge  the  overall  climate.  Even  though 
competition  is  generally  encouraged  in  contracting  efforts,  there  was  some  preliminary  evidence 
that  fund  shortages  and  GAO  accusations  of  duplication  of  research  were  overriding  factors. 
Twenty  of  twenty-nine  interviewees  reported  that  they  were  encouraged  to  promote  competition. 
Eight  of  those  twenty  also  noted  that,  for  them,  competition  only  included  the  process  of  getting 
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three  bids.  Of  the  nine  that  were  not  encouraged  to  promote  competition,  almost  all  cited  budget 
as  the  primary  reason.  However,  two  individuals  noted  that  duplication  of  research  accusations 
(basically  accusation  that  resources  are  being  wasted)  kept  the  competition  down.  No 
distinguishing  factors  effected  this  question. 

Q2-2)  Is  this  ever  done  though  direct  competition  via  multiple  efforts? 

This  question  was  meant  to  delve  further  into  question  2-1.  Its  purpose  was  to  gauge  how  much 
competition  is  encouraged  and  at  what  level.  It  is  also  a  lead-in  to  question  2-3.  Ideally,  by 
getting  respondents  thinking  about  the  benefits  of  letting  multiple  contracts  to  provide  more 
opportunities  for  success,  interviewees  without  experience  in  concurrent  projects  would  think 
about,  rather  than  immediately  dismiss,  question  2.3.  Follow-ups  for  this  question  addressed 
why  multiple  contracts  were  or  were  not  done. 

Almost  two-thirds  of  the  twenty-nine  subjects  indicated  that  multiple  contracts  were  not 
used.  The  basic  themes  of  why  are  broken  out  in  Table  8. 


Table  8  Multiple  Contracts 


Themes 

#  of  responses 

%  of  participants 

#  of  participants 

Yes;  Multiple  contracts. 

10 

34% 

29 

Usually  no;  Multiple  contracts  are  not  let 
due  to  budget  constraints. 

8 

28% 

29 

Usually  no;  Multiple  contracts  are  not  let 
due  to  duplication  of  research/audit  fears. 

11 

38% 

29 

Of  those  that  did  let  multiple  contracts,  all  but  one  restricted  them  to  Phase  I,  or  idea 
generation,  type  contracts.  The  remaining  person  indicated  that  they  occasionally  let  competing 
phase  II,  proof  of  concept,  type  contracts.  All  ten  of  the  individuals  who  responded  yes  indicated 
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that  budget  was  the  primary  reason  for  not  continuing  with  competing  contracts  even  when  it 
might  prove  valuable. 

All  twenty-nine  respondents  did  agree  that  funding  multiple  contracts  and  multiple 
avenues  did  provide  the  best  chance  for  a  quick  success  and  the  best  success.  This  question  was 
definitely  directorate  dependent.  For  SN,  VA,  and  PR,  almost  all  respondents  replied  yes.  In  the 
other  directorates,  the  responses  were  the  opposite. 

Q2-3)  Could  laboratory  development  provide  direct  competition? 

This  is  the  first  question  aimed  at  the  concept  of  concurrency.  Ideally,  those  without  direct 
experience  would  provide  an  opinion  based  on  theory  they  discussed  in  question  2-2  and  at  least 
begin  thinking  about  the  issue  to  answer  later  questions.  Those  with  concurrent  effort  experience 
would  begin  descriptions  here  and  continue  into  the  next  question.  A  typical  follow-up  question 
included:  Would  this  be  beneficial? 

In  this  case,  two-thirds  of  respondents  believed  the  laboratory  could  provide  direct 
competition.  A  basic  breakdown  of  the  themes  is  included  in  Table  9. 


Table  9  Direct  Competition  by  Internal  Effort 


Themes 

#  of  responses 

%  of  participants 

#  of  participants 

Yes;  Internal  efforts  can  provide  direct 
competition. 

19 

66% 

29 

It  might  work 

4 

14% 

29 

No;  Internal  efforts  cannot  provide  direct 
competition. 

6 

20% 

29 

Seven  of  twenty-nine  respondents,  or  approximately  one  quarter  of  the  total  respondents, 
believed  that  concurrent  competitive  development  produced  better,  faster  products  and  more 
capable  government  employees.  Five  believed  that  collaborative  concurrent  development 
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produced  the  best  results.  All  five  described  this  as  a  mix  of  collaboration  and  competition 
where  internal  and  external  development  proceeded  independently,  but  included  meeting  points 
where  infonnation  was  shared. 

Of  the  six  no’s,  three  responded  that  their  directorate  contracting  office  did  not  allow 
direct  competition  with  external  contractors  and  two  cited  fear  of  research  duplication 
accusations.  The  fear  of  accusations  of  duplication  of  research  was  also  cited  by  three 
respondents  answering  that  internal  development  could  provide  direct  competition.  In  all  cases 
this  fear  was  described  as  an  audit  concluding  that  money  had  been  wasted  on  duplicate  efforts 
regardless  of  the  success  of  the  efforts,  but  that  a  failed  effort  in  particular  would  be  targeted  as 
an  example  of  bad  decision  making. 

Again  in  this  question,  the  respondent’s  directorate  influenced  some  responses.  In 
particular,  individuals  from  PR  cited  their  contracting  office  disallowing  a  concurrent 
arrangement. 

Q2-4)  Have  you  been  involved  in  a  concurrent  effort? 

A  yes  answer  to  this  question  was  immediately  followed  with  requests  for  a  full  explanation  of 
the  project,  products,  and  lessons  learned.  Specific  questions  included:  Is  this  method  a  good 
idea? 

Thirteen  of  the  twenty-nine  subjects  had  direct  experience  with  concurrent  development. 
Twelve  of  the  thirteen  thought  this  was  a  better  way  to  go  than  simply  contracting  out  product 
development.  Comments  included  “no  contractor  wants  to  get  beat  by  the  government”  and  “our 
personnel  knew  more  about  the  research  and  were  able  to  ask  tougher  questions  to  the 
contractor.”  Others  pointed  out  that  the  government  was  in  a  position  to  more  easily  try  avenues 
that  were  higher  risk,  but  might  reap  a  bigger  reward.  Seven  of  the  twelve  pointed  out  that  at 
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least  part  of  their  internally  developed  systems  became  a  part  of  the  final  product  and  one 
individual  was  certain  that  the  parallel  internal  effort  was  essential  to  getting  a  working  product 
at  all.  Five  of  the  twelve  individuals  indicated  that  collaborative  concurrent  efforts  where  ideas 
were  shared  throughout  the  process  was  a  better  way  to  go.  All  twelve  indicated  that  taking  the 
best  ideas  from  both  efforts  before  proceeding  to  production  was  the  best  way  to  physically  show 
the  benefits  of  collaborative  efforts,  but  that  the  intangible  benefits  of  training  were  just  as 
important. 

The  one  interviewee  who  thought  the  effort  did  not  work  well  commented  that  the  lack  of 
a  transition  plan  stifled  a  good  effort.  This  individual  recommended  the  involvement  of  the 
program  office  early  to  ensure  transition  planning  occurred. 

Specific  answers  to  this  question  did  not  seem  to  be  dependent  on  individual 
distinguishing  characteristics.  However,  it  is  worth  noting  that  only  one  individual  with  less  than 
ten  years  DoD  laboratory  experience  (out  of  6  included  in  the  sample)  had  experience  with  a 
concurrent  effort. 

Area  Three:  Customer  Relation 

Q3-1 )  How  often  must  you  translate  customer  requests  to  industry? 

This  question  was  meant  to  provide  an  overall  metric  of  how  well  industry  understood  or 
attempted  to  understand  the  requirements  of  the  war  fighter.  It  is  also  intended  to  lay  the 
framework  for  question  3-3.  Depending  on  the  initial  response,  example  follow-up  questions 
might  be:  Do  you  have  problems  getting  customers  to  keep  requirements  stable?  and  Why  do  you 
think  you  have  to  translate  performance  parameters? 

About  half  the  respondents  found  that  they  often  had  to  translate  customer  requests  to 
industry.  A  breakdown  of  the  general  response  themes  is  detailed  Table  10. 
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Table  10  Translation  Periodicity 


Themes 

#  of  responses 

%  of  participants 

#  of  participants 

Often 

14 

48% 

29 

Rarely 

6 

21% 

29 

Never 

3 

10% 

29 

Depends  on  the  customer/Caused  by  the 
customer 

6 

21% 

29 

For  those  that  indicated  they  had  to  translate  requests  often,  two  noted  that  the 
contractor’s  interest  in  cost  and  shareholder  profit  can  be  an  issue.  They  may  be  more  interested 
in  developing  something  that  can  be  modified  for  another  market  than  what  the  government 
actually  needs. 

For  those  that  indicated  they  rarely  had  to  translate  requests,  3  mentioned  enabling 
agreements  and  software.  For  example,  an  Expectation  Management  Agreement  (EM A)  drawn 
up  at  a  meeting  between  customer,  System  Project  Office  (SPO),  and  AFRL  is  beneficial  to  place 
all  requirements  in  perspective  for  easier  translation  to  a  contractor.  Another  example  is  an 
internal  software  product  called  Work  Center  Support  which  allows  contract  managers  and 
contractors  to  communicate  and  collaborate  electronically,  consistently  track  milestone  progress 
rather  than  only  discussing  progress  at  meetings,  and  specify  perfonnance  parameters  to  required 
detail. 

For  those  that  said  never,  one  commented  that  with  an  adequate  budget  and  a  project 
linked  to  on-going  efforts,  communication  was  usually  not  an  issue.  Another  pointed  out  that  a 
close  relationship  helps  tremendously,  but  getting  around  contracting  rules  can  be  an  issue  when 
attempting  to  keep  trusted  partners. 
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Six  individuals  cited  that  the  customer  could  be  the  source  of  communication  problems. 
Specific  comments  included  the  customer  was  too  short-sighted  or  not  sure  of  the  real 
requirement.  Constant  changes  in  AF  policy  were  also  blamed  for  communication  breakdowns. 
None  of  the  distinguishing  factors  of  individual  respondents  seemed  to  influence  responses  to 
this  question. 

Q3-2)  Do  your  industry  contractors  interface  directly  with  the  customers? 

This  question  was  meant  to  gauge  contractor  trust.  Disallowing  customer-contractor  interface 
implies  distrust  of  the  contractor.  It  was  typically  followed  up  with:  Do  you  always  provide  an 
intermediary?  The  results  for  this  question  were  almost  evenly  split  between  yes  and  no.  They 
are  categorized  Table  11. 


Table  11  Direct  Interface 


Themes 

#  of  responses 

%  of  participants 

#  of  participants 

Yes;  Direct,  sometimes  unsupervised, 

11 

38% 

29 

interface  is  frequent. 

Sometimes  direct  interface  is  allowed 

5 

17% 

29 

No;  An  intermediary  is  always  provided 

11 

38% 

29 

Depends  on  the  customer/contractor 

2 

7% 

29 

Comments  associated  with  each  category  were  very  varied  and  seemed  to  be  tied  to  the 
individual’s  past  experience  more  than  anything  else.  Even  though  only  two  of  twenty-nine 
respondents  cited  that  it  depended  on  the  customer  or  contractor,  all  gave  anecdotal  evidence  of 
their  view  based  on  past  experience  with  customers  and  contractors.  The  central  issue  in  the 
evidence  was  trust.  This  included  trust  that  a  customer  would  not  break  contracting  rules  by 
directing  the  contractor  to  perform  tasks  outside  the  contract  scope,  trust  between  contractor  and 
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government  customer  to  completely  share  all  relevant  information,  trust  that  the  contractor  will 
not  try  to  take  advantage  of  direct  access  to  a  customer,  and  trust  that  the  contract  manager  will 
be  informed  of  any  important  contractor-customer  discussions. 

Even  though  the  tabled  results  suggest  almost  40%  always  trust  the  contractor  and  almost 
40%  never  do,  the  results  of  the  anecdotal  evidence  suggest  that  it  depends  upon  the  customer 
and  contractor.  Again  in  this  question,  individual  distinguishing  factors  were  of  little  to  no 
influence. 

Q3-3)  Would  it  be  easier  to  implement  customer  needs  if  the  project  were  internal? 

This  final  question  in  the  customer  relation  area  is  direct.  The  question  was  typically  followed 
up  with:  Why?  Table  12  details  the  basic  responses  to  question  3-3. 


Table  12  Is  Internal  Implementation  Easier? 


Themes 

#  of  responses 

%  of  participants 

#  of  participants 

Yes;  Customer  needs  are  easier  to 
implement  on  internal  projects 

13 

45% 

29 

No;  Customer  needs  are  actually  easier  to 
implement  on  external  projects 

4 

14% 

29 

Does  not  matter 

8 

27% 

29 

Depends 

4 

14% 

29 

The  reasons  given  by  the  thirteen  of  twenty-nine  responders  who  indicated  yes  included 
themes  of  better  control,  more  latitude  for  interpretation,  greater  flexibility  to  implement 
customer  requests,  and  more  trust  that  government  employees  will  keep  the  interests  of  the  war 
fighter  in  mind.  Two  noted  that  external  efforts  tend  to  focus  more  on  what  industry  needs  while 
internal  efforts  tend  to  focus  more  on  what  the  government  needs. 
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Two  of  those  who  said  external  implementation  was  easier  noted  that  internal  efforts 
were  not  generally  held  to  a  plan  or  statement  of  work.  This  led  to  indefinite  efforts  and  lack  of 
accountability.  Two  others  noted  internal  communication  channels  are  difficult  to  navigate 
across  directorates;  this  can  make  it  difficult  to  provide  a  customer’s  need.  One  individual 
brought  up  the  issue  of  requirements  creep  being  more  prevalent  for  an  internal  project  since 
internal  work  is  considered  “free”  by  some. 

Within  the  eight  interviewees  that  responded  it  does  not  matter,  one  claimed  that  up-front 
involvement  of  the  program  office  made  the  difference  in  meeting  customer  needs.  Another 
claimed  that  meeting  customer  needs  is  personally  driven  so  either  internal  or  external  efforts 
could  be  better. 

For  the  four  who  claimed  it  depends,  three  maintained  it  depended  on  the  customer’s 
need.  For  COTS,  large  projects  (large  in  scope  or  large  in  production  run),  and  add-ons  to 
existing  contracted  systems,  external  efforts  could  more  easily  meet  customer  needs.  For  smaller 
short  run  projects  or  truly  innovative  projects,  internal  efforts  were  better  suited.  One  maintained 
that  it  depended  on  the  responsiveness  of  the  contractor.  Distinguishing  factors  for  individuals 
did  not  seem  to  influence  their  response  to  this  question. 

Area  Four:  Capability 

Q4-1)  Are  your  government  employed  personnel  technically  proficient? 

This  question  addressed  the  fear  that,  in  recent  years,  government  engineers  are  becoming 
contract  monitors  and  are  loosing  their  skills  to  really  manage  a  contract  or  perform  any  required 
internal  engineering  and  development.  Other  questions  in  the  capability  and  training  areas  cover 
whether  those  skills  are  really  needed  to  manage  a  contract  (i.e.  question  5-4).  The  typical 
follow-up  question  was:  What  percentage  do  you  think  are  proficient? 
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Contrary  to  this  fear,  most  interviewees  considered  the  majority  of  their  peers  technically 
proficient  in  their  field.  Table  13  breaks  down  the  percentage  of  peers  in  each  interviewee’s 
branch  whom  the  interviewee  assessed  as  technically  proficient. 


Table  13  Percentage  of  Subject’s  Peers  that  are  Technically  Proficient 


Themes 

#  of  responses 

%  of  participants 

#  of  participants 

Less  than  25%  are  technically  proficient 

1 

3% 

29 

Between  35%  and  50%  are  technically 
proficient 

3 

10% 

29 

Between  50%  and  75%  are  technically 
proficient 

5 

17% 

29 

Between  75%  and  100%  are  technically 
proficient 

12 

41% 

29 

100%  of  peers  are  technically  proficient 

8 

28% 

29 

Almost  70%  of  interviewees  rated  75%  or  more  of  their  peers  technically  proficient. 
However,  two  noted  that  many  of  the  most  capable  individuals  were  at  or  close  to  retirement  age 
and  many  of  the  younger  employees  were  not  as  proficient.  This  could  cause  a  decline  in  the 
numbers  over  the  next  few  years.  One  individual  claimed  that  all  personnel  came  into  the  job 
100%  proficient,  but  that  their  skills  atrophied  over  time.  This  argument  is  the  exact  opposite  of 
the  others,  but  still  indicates  a  lack  of  capability  maintenance  which  is  explored  in  the  next  seven 
questions. 

Distinguishing  factors  among  the  individuals  did  seem  to  be  an  issue  in  this  question.  Of 
the  four  that  rated  50%  or  less  of  their  peers  as  not  technically  proficient,  three  of  the  four  were 
on  the  general  management  track  versus  the  technical  track  and  three  of  the  four  had  less  than  ten 
years  of  DoD  laboratory  experience.  The  second  fact  is  more  important  since  the  interviews  only 
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included  six  individuals  with  less  than  ten  years  of  DoD  laboratory  service.  More  than  half  of 
the  new  engineers  in  the  laboratory  deemed  individuals  in  their  branch  not  technically  proficient. 
Q4-2)  Do  they  spend  their  time  developing  systems  or  providing  oversight  to  industry 
developed  systems? 

This  question  builds  upon  question  4-1  and  was  intended  to  address  the  fear  that  government 
employees  are  loosing  their  proficiency  at  engineering  while  becoming  “paper  pushers.” 
Typically,  follow-up  questioning  involved  asking  the  interviewee  to  express  the  ratios  by 
percentage.  A  breakout  of  responses  is  given  Table  14. 


Table  14  Percentage  of  Work  that  is  Industry  Oversight 


Themes 

#  of  responses 

%  of  participants 

#  of  participants 

Less  than  50%  of  the  time  is  spent  on 
oversight  of  contracts  vs.  internal  research 

0 

0% 

29 

Between  50%  and  70%  of  time  is  spent  on 
oversight  of  contracts  vs.  internal  research 

9 

31% 

29 

Between  70%  and  90%  of  time  is  spent  on 
oversight  of  contracts  vs.  internal  research 

13 

45% 

29 

More  than  90%  of  the  time  is  spent  on 
oversight  of  contracts  vs.  internal  research 

7 

24% 

29 

As  the  table  shows,  two  thirds  of  respondents  thought  personnel  in  their  branch  spent 
more  than  70%  of  their  time  managing  contracts  rather  than  doing  actual  research  and  a  full 
quarter  of  respondents  believed  that  over  90%  of  the  time  was  spent  on  paperwork  efforts. 

Oddly,  no  distinguishing  factors  seemed  to  affect  the  response.  The  areas  of  higher  levels 
of  internal  work  (51%  to  70%)  seem  to  be  branch  specific  and  spread  across  the  laboratory 
directorates. 
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Q4-3)  Would  they  be  more  technically  proficient  if  directly  involved  in  system 

development? 

This  question  is  a  follow-up  to  Q4-2.  It  was  meant  to  ascertain  if  hands-on  projects  are  really 
required  for  proficiency  and  if  a  hands-on  level  of  proficiency  is  really  required.  Typical  follow¬ 
up  questions  include:  Is  training  enough?  and  Do  they  need  a  hands-on  level  of  proficiency? 

This  question  also  serves  as  a  control  for  Q5-3  concerning  the  importance  of  experience. 

Twenty-eight  of  the  twenty-nine  subjects  believed  that  hands-on  experience  would 
heighten  technical  proficiency  and  that  training  could  never  substitute  for  the  lessons  of  hands-on 
development.  Four  of  these  twenty-eight  noted  that  mistakes  made  while  performing  hands-on 
research  are  invaluable  learning  tools  that  cannot  be  experienced  any  other  way.  Three  noted 
that  working  directly  on  the  project  gave  an  individual  a  vested  interest  in  the  outcome;  this  is 
much  different  than  someone  simply  managing  a  contract.  The  vested  interest  in  the  outcome 
forced  more  in-depth  learning  and  expertise. 

Only  one  individual  of  twenty-nine  argued  direct  involvement  was  not  needed  to  increase 
technical  proficiency.  This  individual  proposed  that  good,  in-depth,  contractor  management, 
oversight  and  collaboration  could  reap  the  same  benefits  as  direct  involvement  on  an  internal 
project. 

Twenty-six  of  the  twenty-eight  interviewees  who  indicated  that  in-depth  experience 
increased  proficiency  also  thought  that  at  least  some  in-depth  experience  was  important  for 
everyone,  even  the  program  manager,  in  order  to  effectively  do  their  jobs.  The  other  two  thought 
that  some  administrative  positions  may  not  require  a  hands-on  level  of  proficiency.  The 
interviewees  were  in  overwhelming  agreement  on  this  question  so  distinguishing  factors  made 
little  difference. 
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Q4-4)  Would  you  have  to  have  "different"  people  to  develop  a  project  internally? 

This  final  question  in  this  category  again  addresses  the  capability  of  government  employees. 

The  purpose  of  this  question  was  to  provide  a  check  against  Q4-1  where  “technically  proficient” 
may  not  mean  capable  of  internal  development  to  some  individuals.  Follow-up  questions 
include:  What  percentage  would  need  replaced?,  What  percentage  would  need  refreshers  or 
additional  skills  training?,  and  What  percentage  are  capable  as-is? 

This  question  involved  several  tiers  of  response.  Fifteen  of  twenty-nine  interviewees 
believed  that  at  least  some  portion  of  their  branch  workforce  would  need  to  be  replaced  in  order 
to  accomplish  internal  projects.  A  breakdown  of  the  suggested  percentages  is  listed  in  Table  15. 


Table  15  Percentage  of  Workforce  to  Replace 


Themes 

# 

% 

#  of  yes’s 

10%  to  24% 

3 

20% 

15 

25%  to  49% 

7 

47% 

15 

50%  to  70% 

4 

27% 

15 

80% 

1 

6% 

15 

These  numbers  roughly  match  those  from  question  4- 1 .  Only  five  individuals  believed 
that  more  than  50%  of  their  branch  personnel  would  require  replacement  in  order  to  accomplish 
internal  development.  Three  individuals  included  the  caveat  in  their  response  that  they  were 
only  talking  about  civilian  employees.  They  thought  that  a  higher  percentage  of  military 
personnel  would  require  replacement  with  more  proficient  individuals.  Two  individuals  also 
noted  that  the  numbers  may  change  in  the  near  future  due  to  the  retirement  of  capable 
individuals.  In  addition  to  replacement,  some  individuals  thought  engineering  “refresher 
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training”  would  be  required  for  a  percentage  of  those  not  replaced.  This  is  summarized  in  Table 


16. 


Table  16  Percentage  of  Workforce  Requiring  Refresher  Training 


Themes 

#  of  responses 

30%  to  49% 

5 

50%  to  99% 

2 

100% 

1 

Fourteen  of  twenty-nine  interviewees  indicated  that  no  replacement  was  required.  They 
believed  assigned  personnel  were  capable  of  internally  developing  a  project.  Three  of  the 
fourteen  thought  that  the  branch  only  needed  a  target  or  goal  and  the  personnel  would  gain  the 
required  knowledge  (if  necessary)  to  achieve  it.  Two  of  the  fourteen  commented  that  moving  or 
teaming  personnel  was  the  only  requirement  for  success.  However,  six  of  these  fourteen  thought 
that  even  though  no  replacement  was  required,  at  least  some  engineering  refresher  courses  would 
be  required. 

Eleven  of  twenty-nine  interviewees  stated  that  additional  personnel  would  be  required  for 
internal  development  regardless  of  the  percentage  of  people  requiring  replacement.  The 
directorate  of  the  interviewee  seemed  to  be  a  distinguishing  factor  in  this  question.  A  large 
portion  of  the  negative  responses  came  from  HE  and  SN,  while  a  large  portion  of  the  high 
replacement  or  refresher  training  requirements  came  from  VA,  IF,  and  PR. 

Area  Five:  Training 

Q5-1)  How  do  you  ensure  your  engineers/technicians  remain  technically  proficient? 
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The  first  question  of  this  set  is  meant  to  address  any  other  ways  technical  people  can  remain 
capable  without  direct  hands-on  research  projects.  Twenty-seven  of  twenty-nine  interviewees 
indicated  that  the  primary  means  of  remaining  technically  proficient  is  self-initiative.  There  is  a 
requirement  to  maintain  a  level  of  APDP  certification  and  eighty  hours  of  credits  every  two 
years,  but  the  variety  of  training  that  counts  for  credit  allows  an  individual  to  set  their  own 
development  path.  One  individual  stated  that  this  was  not  significant  and  insisted  that  if  the 
laboratories  provide  interesting  (hands-on)  work,  the  employees  will  ensure  that  their  skills 
current.  Eleven  individuals  noted  that  training  (including  advanced  degrees,  conferences,  and 
paper  publication)  is  encouraged.  Four  individuals  indicated  that  Individual  Development  Plans 
(IDPs)  were  supposed  to  be  discussed  between  employee  and  supervisor  to  ensure  adequate 
training,  but  twenty-five  of  twenty-nine  interviewees  did  not  mention  them.  Despite  question  4- 
3,  only  three  individuals  indicated  that  rotating  hands-on  projects  and  field  work  around  the 
office  was  part  of  their  training  program.  One  noted  that  they  were  going  to  add  hands-on 
projects. 

Two  of  the  twenty-nine  interviewees  indicated  that  their  directorate  had  a  formal 
mentorship  or  formal  on-the-job  training  (OJT)  plan.  Employees  in  these  areas  were  managed 
based  on  their  track  to  ensure  at  least  minimum  proficiencies  were  retained  as  needed.  These 
individuals  were  from  two  separate  directorates.  No  other  interviewee  from  either  directorate 
mentioned  the  programs  which  implies  that  none  of  the  individual  distinguishing  factors  are 
correlated  with  responses. 

Q5-2)  Is  training  centered  on  engineering  or  contract  management? 

Typically,  this  question  is  followed-up  with:  Is  required  training  centered  on  engineering  or 
contract  management?  This  question  is  intended  to  determine  the  relative  importance  of 
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engineering  vs.  contract  management.  In  other  words,  are  the  laboratories  deliberately  training 
contract  managers  rather  than  engineers  or  vice  versa? 

The  answers  to  this  question  were  also  varied  and  multi-tiered,  but  for  the  root  question 
twenty-one  of  twenty-nine  interviewees  indicated  that  required  training  was  centered  on  contract 
management.  This  training  includes  Acquisition  Professional  Development  Program  (APDP)  up 
to  level  three.  Three  people  thought  that  this  APDP  training  was  designed  for  program  managers 
at  SPOs  and  virtually  useless  in  the  AFRL  environment.  Eight  of  those  twenty-one  who  said 
required  training  was  centered  on  contract  management  also  said  that  most  people  choose 
engineering  training  for  their  voluntary  portion  to  fulfill  the  eighty  hours  every  two  years 
requirement.  One  individual  noted  how  fast  technology  was  advancing  and  commented  that 
required  training  should  be  centered  on  engineering. 

Six  of  the  twenty-nine  interviewees  indicated  that  both  engineering  and  contract 
management  were  equally  stressed.  Two  commented  that  individuals  were  encouraged  to  get 
smart  in  the  areas  they  were  lacking. 

Two  of  the  twenty-nine  interviewees  stated  that  the  answer  to  the  question  is  job 
dependent.  Personnel  in  administrative  and  contract  management  jobs  are  encouraged  to  take 
more  contracting  classes  and  those  in  technical  and  hands-on  jobs  are  encouraged  to  take  more 
engineering  classes. 

In  most  cases,  individual  distinguishing  factors  were  not  significant.  However,  all  the 
respondents  from  PR  indicated  that  both  contract  management  and  engineering  were  equally 
stressed.  One  comment,  that  personnel  were  encourage  to  gain  knowledge  in  their  deficient 
areas,  came  from  an  interviewee  from  PR.  In  addition,  one  PR  response  noted  that  employees 
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were  also  encouraged  to  take  business  courses  where  appropriate  to  ensure  basic  business 
knowledge. 

Q5-3)  In  your  opinion,  what  can  be  done  to  improve  the  situation? 

This  question  was  included  to  see  if  any  individuals  recommended  hands-on  training  or  internal 
development  as  a  training  method  to  ensure  capable  employees.  Three  out  of  twenty-nine 
interviewees  actually  indicated  that  this  is  the  case.  The  remaining  responses  were  also  very 
informative.  All  responses  are  presented  in  Table  17. 

Based  on  questions  4-3  and  5-1,  more  recommendations  for  hands-on  training  were 
expected.  Interviewees  obviously  felt  more  strongly  about  these  other  proposals.  There  were  no 
obvious  links  between  any  distinguishing  factors  and  individual  responses  to  this  question. 

Table  17  Training  Improvement  Responses 

Theme _ #  respondents 


Develop  an  AFRL  specific  acquisition/system  engineering/DAU  training  7 

program. 

Mandate  that  a  portion  of  80hrs/2yrs  be  used  for  technical  proficiency  5 

refresher  courses. 

Need  an  all-in-one  training  management  system  5 

Keep  the  content  local,  focused  and/or  short  (a  one  day  class  or  seminar  for  4 

example  in  neural  nets  or  a  briefing  on  war  fighter  lessons  learned). 

Engineers  want  efficient  training,  not  trips. 

Reduce  non-technical  "mandatory"  training.  4 


Need  to  be  with  the  customer  or  have  access  to  the  customer's  training  to  help  3 

understand  what  they  need  (we  are  blocked  from  some  insightful  classes  due 
to  career  field  requirements). 

VA's  mentoring  program  and  checklist  or  similar  formalized  system  should  be  3 

used  across  the  laboratory;  management  and  leadership  should  be  heavily 

involved. 
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Continuation  of  Table  17  Training  Improvement  Responses 


Theme 

#  respondents 

Allow  more  time  for  training. 

2 

Training  points  of  contact  and  supervisors  need  to  do  a  better  job  advertising 
available  training  courses  and  recommending  alternative  courses. 

1 

Need  a  course  on  how  to  apply  an  appropriate  level  of  systems  engineering 
depending  on  the  projects  (not  all  projects  need  the  full  blown  300,000K 
effort);  this  needs  to  be  consistent  throughout  the  laboratory  and  for  program 
reviews. 

1 

Put  off  the  acquisition  courses  for  later  (unless  a  very  specific  one  can  be 
developed  for  laboratory  personnel);  concentrate  on  technical  training,  design 
of  experiments,  testing,  and  statistical  analysis  for  the  first  couple  of  years  at 
least. 

1 

Talk  to  people  to  see  what  they  need;  make  the  reasons  for  any  mandatory 
training  clear  to  the  trainee. 

1 

Provide  branch  chiefs  and  directorate  heads  with  management  and 
administration  skills  training. 

1 

Q5-4)  Do  government  personnel  really  need  engineering  experience  or  engineering 
degrees  in  today 's  environment? 

This  final  training  question  was  intended  to  gauge  the  importance  of  engineering  experience  to 
laboratory  personnel  capability.  Results  pointing  to  the  requirement  of  a  technical  degree  were 
of  secondary  importance.  Typical  follow-up  questions  included:  Would  a  business  or  other 
degree  work  just  as  well  with  experience?  and  Which  is  more  important  -  the  degree  or  the 
experience? 

Twenty-six  of  twenty-nine  interviewees  stated  that  an  engineering  degree  was  essential  in 
today’s  environment.  Of  those  twenty-six,  seventeen  thought  the  degree  was  more  important 
than  experience.  Eight  of  the  twenty-six  thought  both  were  equally  important,  especially  at 
higher  levels  in  the  chain.  One  of  the  twenty-six  thought,  even  though  the  degree  was  a  must, 
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experience  was  more  important,  especially  for  someone  on  a  technical  track.  Reasons  for  the 
degree  requirement  fell  under  four  basic  categories  and  are  detailed  Table  18. 

Some  individuals  had  very  strong  feelings  and  comments  like  “it  can’t  be  done  without 
an  engineering  degree”  and  “it  is  too  easy  to  let  a  contractor  make  bad  decisions  without  an 
engineering  degree”  were  typical.  Four  individuals  clearly  indicated  that  not  only  was  an 
engineering  degree  required  to  effectively  manage  a  contract,  but  due  to  fast  moving  technology, 
an  engineering  degree  specific  to  the  type  of  research  being  contracted  was  essential.  One 
individual  commented  that  “if  the  contact  is  general  enough  that  a  non-technical  person  can 
handle  it,  then  it  doesn’t  belong  in  the  laboratory”  and  another  commented  “there  is  a  difference 
between  a  contract  monitor  and  a  contract  manager  and  a  person  without  an  engineering  degree 
is  almost  always  just  a  monitor.” 


Table  18  Reasons  for  an  Engineering  Degree 


Themes 

#  of  responses 

Accuracy/ Ability  to  Manage  rather  than  just 
Monitor  contracts 

11 

Keep  the  contractor  honest 

6 

Credibility/Communication 

3 

Accountability 

2 

Three  of  twenty-nine  interviewees  stated  that  it  does  not  require  an  engineering  degree  to 
adequately  manage  a  laboratory  contract  in  today’s  environment.  All  three  of  these  individuals 
believed  experience  was  more  important  than  the  degree.  In  addition,  some  specific  comments 
were  “sometimes  a  business  background  is  important”  and  “we  need  other  points  of  view.” 
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Oddly,  comments  referring  to  business  skills  were  also  given  as  caveats  by  three 
individuals  that  thought  a  degree  was  required.  The  comments  indicated  that  higher  level 
managers  need  basic  business  skills  or  training  in  addition  to  their  engineering  degree.  Again, 
individual  distinguishing  factors  did  not  seem  to  influence  responses  to  this  question. 

Area  Six:  Cost 

Q6-1)  Would  it  cost  more  to  develop  internally? 

This  first  question  delves  into  the  issue  of  cost.  Typical  follow-up  questions  included  Why?  and 
Would  it  be  cheaper?  As  stated  in  the  literature  review  and  as  one  interviewee  put  it  “Just  the  act 
of  letting  a  contract  is  expensive.”  The  general  consensus  among  the  seven  interviewees,  who 
discussed  the  actual  cost  of  a  contracting  effort,  was  that  letting  a  single  contract  contained  total 
costs  of  around  $300,000  (time,  personnel,  paperwork,  etc). 

Only  five  of  twenty-nine  interviewees  thought  it  would  cost  more  to  develop  internally. 
Reasons  given  included  lack  of  a  timeline  and  loose  requirements  which  caused  long  production 
schedules  and  requirements  creep,  loss  of  capability  which  increases  costs  over  time,  and  internal 
mistakes  which  the  contractor  charges  more  to  fix.  One  individual  also  noted  that  “assuming  the 
project  is  long  enough  to  make  back  the  initial  investment  in  time  to  contract,  we  are  buying  time 
to  spend  on  other  projects  which  saves  overall  costs.”  Four  of  the  twenty-nine  interviewees 
claimed  that  it  depends  on  such  matters  as  scope,  required  integration,  laboratory  equipment 
upgrade  requirements,  and  how  the  money  is  counted. 

Twenty  of  twenty-nine  respondents  stated  that  it  actually  costs  less  to  develop  internally. 
Four  individuals  stated  that  for  small  efforts,  the  cost  to  let  a  contract  can  often  cost  more  than 
the  contract  is  worth.  Five  individuals  stated  that  external  overhead  and  rates  have  grown  to 
match  or  exceed  any  internal  overhead  and  that  even  efficient,  large  companies  sometimes 
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squander  contract  money  with  little  results.  One  individual  stated  that  it  might  cost  more 
initially,  but  over  the  long  term  internal  development  would  drive  competition,  eliminate  excess 
profit,  and  provide  a  risk  reducer  for  bigger  initiatives.  Two  individuals  did  note  that  an  internal 
schedule  was  important  and  transition  costs  could  be  an  issue  if  transition  is  not  adequately 
planned  during  the  effort.  Again,  in  this  question,  individual  distinguishing  factors  did  not  seem 
to  influence  responses. 

Q6-2)  What  tradeoffs  would  be  required  to  develop  internally? 

This  question  was  targeted  at  activities  that  might  fall  through  the  cracks  if  the  laboratory  shifted 
to  a  high  percentage  of  internal  development.  Despite  the  question  testing  with  SMEs  mentioned 
in  methodology,  this  question  still  produced  considerable  confusion.  Many  individuals  began 
giving  responses  similar  to  those  in  the  feasibility  area  concerning  what  they  required.  Although 
these  responses  were  considered  and  recorded  to  see  if  any  new  information  could  be  gained  for 
area  one  questions,  they  are  not  included  here.  Instead,  when  confusion  occurred,  the  following 
follow-up  question  was  asked:  Is  there  anything  you  would  no  longer  be  able  to  do?  The  results 
were  extremely  varied;  however,  the  main  themes  are  detailed  in  Table  19. 

Within  the  fourteen  responses  for  better  prioritization,  comments  included  the  ability  to 
quickly  cut  projects  that  were  not  producing  results  and  included  risk  reduction  within 
prioritization.  Seven  of  these  fourteen  individuals  indicated  that  even  when  useless  and  very 
high  risk  programs  were  cut  after  prioritization,  some  worthwhile  programs  would  still  need  to 
be  eliminated  due  to  lack  of  resources. 

Those  that  thought  that  flexibility  would  need  to  be  reduced  were  generally  concerned 
that  the  advantages  provided  though  internal  flexibility  in  meeting  customer  requirements  caused 
the  typical  internal  effort  to  proceed  very  slowly  and  be  highly  vulnerable  to  requirements  creep. 
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This  was  seen  as  different  from  typical  contract  theory  in  which  the  product  quality  is 
proportional  to  the  amount  of  funding  contributed  and  the  contractor  is  held  accountable  for 
production.  Recommended  solutions  to  this  problem  were  ensuring  the  internal  effort  included  a 
timeline,  a  set  of  goals,  and  measures  of  efficiency  and  then  holding  members  of  the  effort 
accountable  for  meeting  those  goals. 

Table  19  Tradeoffs  Required  for  Internal  Development 


Themes 

#  of  responses 

No  tradeoffs  are  required. 

5 

We  would  need  to  better  prioritize  and 
possibly  cut  the  number  and/or  reduce  the 
scope  of  programs  under  development. 

14 

We  would  need  to  reduce  internal  flexibility 
in  order  to  ensure  projects  were  complete  by 
managing  internal  efforts  like  a  contract. 

8 

Management  and  regulatory  control  of 
budget,  manpower,  and  workforce  must  be 
reduced. 

8 

Assign  a  greater  emphasis  to  short  term 
payoffs  rather  than  long  term  research. 

3 

Reduce  the  level  of  detail  required  in  internal 
acquisition  plans. 

1 

Those  that  stated  management  control  reduction  was  essential  argued  that  to  successfully 
compete  with  a  contractor,  the  ability  to  transfer,  hire,  fire,  and  team  personnel  needed  to  reside 
with  those  leading  the  projects.  This  would  ensure  that  the  personnel  possess  the  skills  required 
to  complete  the  effort.  If  the  personnel  do  not  have  the  required  skills,  more  time  and  effort  will 
be  devoted  to  training  personnel  in  the  skills  required  than  to  simply  contract  out  the  effort. 
Communication  was  also  noted  as  an  issue.  Management  control  of  cross  chain  of  command 
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communication  required  reduction  so  that  project  leaders  could  easily  reach  experts  needed  to 
complete  efforts.  One  individual  even  suggested  further  laboratory  consolidation  to  allow  for 
easier  teaming  and  communication.  Finally,  two  individuals  noted  that  Congressional  earmarks 
often  required  certain  contracts.  While  this  is  included  as  a  management  control  that  must  be 
reduced,  it  is  outside  the  scope  of  the  laboratories. 

The  only  individual  distinguishing  factor  of  note  was  that  all  five  individuals  who 
believed  no  tradeoffs  were  required  were  from  the  general  management  versus  technical  track. 
However,  this  only  accounts  for  about  one-third  of  the  individuals  interviewed  in  this  track. 

Q6-3)  How  would  concurrent  development  apply  to  or  affect  your  area? 

Unlike  earlier  questions  about  concurrent  development,  this  question  was  specifically  directed  at 
cost.  Ideally,  individuals  would  build  on  earlier  concurrent  questions  and  describe  any  cost 
savings  gained  via  better  or  more  quickly  developed  products  which  might  offset  the  price  of 
dual  efforts.  Follow-up  questions  included:  Would  a  competitive  internal  program  force  down 
contract  costs?  and  Specifically,  how  would  it  affect  cost?  Table  20  summarizes  the  major 
themes  from  respondents. 


Table  20  Concurrent  Development 


Themes 

#  of  responses 

It  provides  better  products  and  a  more 
skilled  internal  workforce  which  offsets 
any  duplication  of  effort  costs. 

15 

It  reduces  cost. 

13 

It  is  not  allowed  or  cannot  be  done. 

8 

Seven  of  twenty-nine  interviewees  commented  that  concurrent  efforts  could  both  reduce 
both  cost  and  improve  products.  Another  interesting  fact  was  that  five  of  the  twenty-one 
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individuals  who  thought  concurrent  efforts  were  beneficial,  believed  collaborative  concurrent 
efforts  (where  knowledge  was  shared  between  the  teams  at  points  during  development)  was 
superior  to  competitive  concurrent  efforts.  Two  general  comments  from  the  twenty-one 
individuals  were  “a  good  management  process  is  required  to  ensure  we  reap  the  benefits  of 
trying  something  different,  getting  the  experience,  and  managing  the  contract  well  all  at  once” 
and  “solid  requirements  are  a  must  for  both  internal  and  external  efforts  for  this  to  work.” 

Within  the  thirteen  respondents  who  thought  that  concurrent  efforts  reduced  costs,  five 
noted  that  contract  mangers  gained  a  better  understanding  of  the  costs  by  being  involved  in  a 
concurrent  effort  which  could  lead  to  better  decision  about  contract  costs.  Four  of  the  thirteen 
noted  that  internal  efforts  have  a  better  understanding  of  what  is  specifically  needed  and  are 
designing  to  that  need,  while  contractors  are  often  trying  to  sell  either  commercially  viable 
solutions  or  all-inclusive  (extra  “bells  and  whistles”)  solutions  which  usually  cost  and  include 
more  than  the  government  needs.  Two  of  the  thirteen  thought  that  by  exploring  higher 
risk/higher  payoff  solutions  via  internal  efforts,  the  laboratory  saved  money  on  contractor  costs 
(that  generally  charge  more  for  high  risk  endeavors)  and  potentially  reduced  cost  by  finding  a 
higher  payoff  solution.  Two  individuals  noted  that  a  contract  manager  who  participates  in  a 
concurrent  effort  has  a  better  understanding  of  the  research  and  more  credible  grounds  to 
threaten  a  work  stoppage  if  the  contractor  is  wasting  money  or  not  adequately  progressing. 
Finally,  two  individuals  claimed  that  internal  effort  successes  helped  sell  AFRL  as  a  “brand”  and 
provided  the  grounds  for  cost  reduction  by  allowing  negotiations  with  the  terms  “if  you  [the 
contractor]  won’t  do  it  for  less  than  that,  we  [AFRL]  will  do  it  ourselves.” 

The  fifteen  who  thought  that  concurrent  efforts  led  to  better  products  noted  that  increases 
in  laboratory  personnel  skills  and  understanding  allowed  better  questions  and  more  focused 
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management  of  the  contractor,  that  internal  research  on  higher  risk  alternatives  allowed  bigger 
payoffs,  and  that  internal  solutions  can  be  the  better  product.  Seven  of  the  fifteen  claimed 
concurrent  development  was  a  great  way  to  increase  internal  personnel  skill  levels.  Most  agreed 
that  the  best  of  both  internal  and  external  efforts  should  be  used  for  final  production  to  ensure  a 
better  overall  product. 

Of  the  eight  individuals  who  claimed  concurrent  efforts  were  not  possible,  three  claimed 
their  budget  or  allegations  of  research  duplication  made  a  concurrent  effort  impossible,  three 
claimed  that  their  contracting  section  prohibited  concurrent  efforts,  and  two  claimed  that  internal 
politics  made  teaming  the  required  skills  and  people  to  manage  a  concurrent  effort  impossible. 
No  one  claimed  that  concurrent  efforts  would  not  reduce  cost  or  produce  better  products. 

The  only  individual  distinguishing  factor  that  was  significant  was  if  an  interviewee  was 
from  PR.  All  three  responses  indicating  contracting  could  not  allow  an  internal  effort  came  from 
PR. 

Q6-4)  If  your  product  was  internally  developed,  would  it  be  cheaper  to  produce  or 

sustain? 

This  question  was  designed  to  explore  patent  rights  and  proprietary  technology  generally 
associated  with  contractor  versus  government  developed  programs.  If  interviewees  initiated  this 
topic  in  response  to  earlier  questions,  the  responses  were  transposed  to  here.  A  typical  follow-up 
question  included:  If  your  product  was  originally  contractor  developed,  would  it  be  cheaper  to 
produce  or  sustain?  The  goal  was  to  compare  long-term  sustainment  costs  based  upon  the 
original  design  source  of  the  product.  Even  with  the  SME  question  validation  discussed  in 
methodology,  the  phrasing  of  this  question  confused  several  individuals  who  attempted  to 
address  government  versus  contractor  sustainment.  Thus,  for  some  subjects  the  qualifier, 
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regardless  of  whether  government  or  a  contractor  performs  the  sustainment,  was  included  and 
the  question  repeated  to  illicit  a  meaningful  answer.  The  responses  are  detailed  Table  21. 


Table  21  Internal  vs.  External  Development  Concerning  Sustainment 


Themes 

#  of  responses 

%  of  participants 

#  of  participants 

External  development  leads  to  cheaper 
sustainment  costs. 

11 

38% 

29 

Internal  development  leads  to  cheaper 
sustainment  costs. 

5 

17% 

29 

Does  not  matter 

5 

17% 

29 

Depends 

8 

28% 

29 

For  those  eleven  who  thought  that  external  development  led  to  cheaper  sustainment  costs, 
two  insisted  that  industry  would  either  charge  more  to  produce  or  block  an  internal  effort  from 
production  in  order  to  ensure  they  received  their  development  cut.  Four  of  the  eleven  claimed 
that  contractors  would  be  producing  and  sustaining  products  in  the  long  run  and  that  the  cost  of 
transitioning  products  from  internal  development  to  external  production  outweighed  any  cost 
gains  of  internal  development.  One  individual  noted  that  this  flaw  could  be  overcome  by 
including  a  contractor  associated  with  the  eventual  production  contract  on  the  in-house 
development  team.  Finally,  four  of  the  eleven  pointed  out  that  internal  efforts  typically  forget  to 
design  with  production  and  sustainment  in  mind,  so  contractors  are  still  required  to  rework  the 
product.  However,  one  interviewee  commented  that  the  laboratories  are  considering  the  addition 
of  a  Manufacturing  Readiness  Level  (MRL)  requirement  to  force  development  efforts  to  consider 
manufacturing  in  design. 


72 


Of  the  five  that  indicated  internal  development  efforts  could  produce  cheaper  long-run 
sustainment  costs,  two  interviewees  noted  that  the  internal  development  process  must  consider 
sustainment  during  the  design  for  this  to  work.  One  also  noted  that  to  gain  these  benefits  a  stable 
design  team  is  required  until  the  project  is  finished. 

The  eight  individuals  who  stated  that  it  depends  were  split  into  three  categories.  Two 
stated  it  depends  on  the  product.  Specific  and  small  projects  were  better  suited  for  an  internal 
team  while  large  or  on-going  projects  were  better  suited  for  an  external  team.  Two  stated  that  it 
depends  on  who  will  sustain  the  product.  If  the  government  plans  to  sustain  the  product,  then 
development  should  be  internal.  If  the  product  will  be  contractor  sustained,  then  development 
should  be  external.  Three  cited  that  it  depends  on  the  contractor.  If  the  contractor  is  willing  to 
use  our  designs,  then  an  internal  effort  can  produce  lower  sustainment  cost  products. 

Of  the  five  that  thought  it  did  not  matter,  two  individuals  noted  that  regardless  the 
product  designer,  sustainment  costs  may  be  influenced  by  SPOs  whose  guidance  is  often  to 
concentrate  on  the  best  performance  characteristics  for  the  initial  buy  rather  than  an  attempt  to 
decrease  long  term  sustainment  cost.  In  this  question,  individual  distinguishing  factors  did  not 
seem  to  influence  responses. 

Area  Seven:  Speed  to  War  Fighter 

Q7-1)  Could  an  internally  developed  system  be  fielded  faster  than  an  externally 
developed  one? 

The  first  question  of  this  series  was  directed  at  overall  time  to  field.  Conceivably,  faster  work 
implies  faster  development  spirals  and  better  products  to  the  war  fighter  faster.  Responses  were 
generally  consistent  with  this  category  and  did  not  require  follow  up  questions. 
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Responses  to  this  question  were  approximately  evenly  split.  Nine  of  twenty-nine 
interviewees  responded  that  internally  developed  systems  could  be  fielded  faster  than  externally 
developed  ones.  Two  people  noted  flexibility  as  the  key  to  internal  efforts.  One  noted  that  the 
time  it  takes  to  let  a  contract  can  cause  significant  delay  in  product  development.  Finally,  one 
noted  that  rapid  field  testing  was  more  easily  available  to  internal  efforts. 

Nine  of  the  twenty-nine  interviewees  believed  externally  developed  systems  could  be 
fielded  faster  than  internally  developed  ones.  Five  individuals  cited  integration  with  existing 
systems  as  the  primary  reason.  They  claimed  that  most  laboratory  development  efforts  would  be 
integrated  into  something  already  contracted  so  that  contractor  should  perform  the  development 
of  the  new  system.  Two  individuals  claimed  that  contractors  are  able  to  assign  one  person  to  one 
technology  while  internal  design  often  has  one  person  assigned  to  multiple  development  efforts. 
Two  interviewees  believed  contractors  had  better  networks  with  SPOs  and  war  fighters  pulling 
the  system  to  get  it  fielded  faster.  One  person  commented  that  the  internal  metal  gap  (not  the 
physical  division  of  funding  streams)  between  individuals  concentrating  on  6.2  research  and 
individuals  concentrating  on  6.3  development  hindered  internal  efforts  to  develop  products. 

Eight  of  twenty-nine  subjects  thought  that  both  methods  were  equally  expeditious.  Two 
of  these  indicated  that  internal  development  was  actually  faster,  but  transition  would  require 
additional  time.  However,  another  individual  stated  that  the  DoD  has  a  problem  transitioning 
both  internal  and  external  development  efforts. 

Three  of  twenty-nine  interviewees  thought  that  the  response  depended.  Two  cited  the 
sustainment  effort  as  the  differentiating  factor  with  internal  favoring  small  sustainment  efforts 
and  external  favoring  large  ones.  One  claimed  it  depended  upon  who  owned  the  process  experts. 
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If  the  expert  was  a  laboratory  employee  then  the  laboratory  would  be  faster.  If  the  expert  was  in 
industry,  then  industry  would  be  faster. 

Again,  individual  distinguishing  factors  did  not  seem  to  influence  responses  to  the 
question.  However,  some  interviewees  went  more  in-depth  on  the  issues  associated  with 
transition.  None  thought  the  actual  division  of  money  within  the  PPBS  system  presented  an 
issue  if  transition  was  planned  correctly.  Issues  with  transition  other  than  funding  are  outside  the 
scope  of  this  thesis,  but  will  be  briefly  addressed  in  chapter  five. 

Q7-2)  Can  the  government  perform  tests  that  are  required  for  fielding  faster  than 

industry? 

This  question  is  specifically  directed  at  testing — whether  it  can  be  performed  faster  internally  or 
externally.  This  question  was  originally  intended  to  help  detennine  if  processes,  paperwork,  or 
PPBS  funding  constraints  were  causing  fielding  issues.  However,  responses  to  question  7-1 
essentially  eliminated  funding  stream  divisions  as  an  issue  for  internal  development.  Responses 
to  this  question  usually  did  not  require  subsequent  questions. 

Nineteen  of  twenty-nine  interviewees  believed  that  nether  internal  or  external  efforts  had 
a  conclusive  advantage  in  systems  testing.  Twelve  of  twenty-nine  respondents  concluded  that 
neither  selection  was  better,  although  two  of  those  commented  that  internal  testing  might  be 
cheaper.  Four  of  twenty-nine  respondents  claimed  the  response  depends  upon  who  employs  the 
expert  and  owns  the  test  facilities.  Three  of  twenty-nine  respondents  believe  that  testing  is 
always  a  joint  effort. 

The  other  ten  of  twenty-nine  respondents  were  almost  evenly  divided.  Four  of  twenty- 
nine  interviewees  claimed  that  internal  efforts  had  the  advantage  in  testing.  Two  claimed  that 
access  to  military  personnel,  who  must  take  a  testing  risk  made  the  process  faster  and  cheaper 
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than  a  contracted  test.  Six  of  twenty-nine  interviewees  claimed  that  external  efforts  had  the 
advantage  in  testing.  Comments  indicated  the  belief  that  the  contractor  may  have  more 
experience  and  a  better  network  for  conducting  tests  or  the  government  no  longer  has  the 
capability  to  conduct  tests.  Of  the  respondents  that  believed  the  government  or  industry  had  a 
conclusive  advantage,  individual  distinguishing  factors  did  not  seem  to  influence  responses  to 
the  question. 

Q7-3)  What  roadblocks  do  internally  developed  systems  or  industry  developed  systems 

have  an  advantage  in  surmounting  to  reach  IOC? 

The  final  question  in  this  area  was  intended  to  exhaust  the  list  of  items  differentially  affecting 
internal  and  external  development  fielding.  Follow-up  questions  were  not  generally  required. 
Responses  were  widely  varied  and  are  presented  in  Table  22. 


Table  22  Roadblocks 


Government  Effort 

Contractor  Effort 

Risk  (3) 

Integration  with  the  contractor’s  existing 
system  is  easier  (7) 

CD 

bD 

Cost 

-5 

3 

Network/Political  Expectations  (3) 

a 
> 
i  r  < 

Funding  or  requirements  changes  are 

< 

easier 

Ease  of  sub-contracting/Can  quickly  spend 
end  of  year  money  (2) 

Certification 

c n 

CD 

bO 

Problems  with  Transition/Supportability 

Stakeholder  and  outside  sales  requirements/ 

cd 

3 

(10) 

Requirement  to  make  a  commercially  viable 

cd 

> 

system  and  make  money  (3) 

’"O 

cd 

C/3 

3 

Requirements  Creep 

Accountability 
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All  ten  individuals  citing  transition  as  an  issue  were  asked  if  the  division  of  funding 
streams  created  the  problem.  Two  individuals  noted  that  the  division  aggravated  the  situation 
and  direct  access  to  6.3b  and  6.4  funds  within  the  laboratory  would  help,  but  all  ten  stated  that  if 
planned  correctly,  funding  stream  divisions  would  not  affect  transition.  One  individual  noted 
that  the  key  to  avoiding  transition  roadblocks  for  internal  and  external  development  efforts  was  a 
well  managed  experienced  team. 

The  seven  that  stated  industry  efforts  had  an  advantage  when  integrating  a  new 
development  with  an  existing  contracted  system  all  included  the  caveat  that  this  is  only  an 
advantage  when  the  contractor  developing  the  new  capability  is  the  same  contractor  that  owns 
the  current  system.  Any  other  contractor  would  usually  not  hold  an  advantage  over  the 
government. 

Five  of  the  twenty-nine  interviewees  stated  there  were  no  roadblocks  to  Initial 
Operational  Capability  (IOC)  that  either  the  government  or  industry  had  an  advantage  in 
surmounting.  Two  interviewees  stated  that  specific  expert  individuals  might  give  one  or  the 
other  an  advantage  on  some  projects  and  one  interviewee  stated  that  a  well  managed  expert  team 
was  the  key  either  way.  Once  again,  no  individual  distinguishing  factors  seemed  to  influence 
responses  to  the  question. 

Area  Eight:  Guidance 

Q8-1)  What  guidance  do  you  follow  when  deciding  what  to  develop  internally  and  what 

to  contract  to  industry? 

Area  eight  is  the  first  area  not  directly  related  to  outsourcing.  However,  as  discussed  in  the 
literature  review,  company  guidance  can  affect  actions  and,  therefore,  responses  to  most  of  the 
other  questions.  Logic  dictates  that  this  would  be  even  more  true  in  a  military  organization  built 
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on  precisely  following  orders  and  chain  of  command.  This  question  was  directed  at  physical 
regulations  or  memorandums  to  discover  if  laboratory  personnel  were  being  actively  pressured  to 
contact  out  most  projects.  As  was  presented  in  Chapter  2,  Preliminary  evidence  and  past 
commander  statements  indicate  this  is  true.  When  the  question  was  asked,  many  individuals 
immediately  began  reciting  rules  of  thumb.  In  this  case,  the  responses  were  considered  and 
recorded  under  question  8-2.  The  question  was  then  further  explained  and  restated. 

The  lack  of  instruction  awareness  revealed  here  is  of  importance.  Twenty-six  of  twenty- 
nine  respondents  were  not  aware  of  any  guidance.  One  individual  noted  that  they  used  the 
contracting  rule  to  acquire  COTS  products  whenever  possible,  two  individuals  noted 
Congressional  eannarks  on  funds  required  some  funding  to  be  spent  on  specific  contracts,  and 
one  individual  mentioned  contracting  decisions  would  be  reviewed  by  a  supervisor.  Although 
the  80%/20%  contract  versus  internal  development  rule  from  AFRL’s  past  commander,  General 
Paul,  was  quoted  a  number  of  times  by  individuals  during  their  interview,  only  one  individual 
mentioned  it  here.  Responses  to  this  question  effectively  show  that  the  decision  to  contract  out 
or  not  is  left  to  the  individual. 

Q8-2)  Do  you  have  any  unwritten  rules  on  the  subject? 

This  is  the  only  follow-up  question  to  Q8-1.  In  addition  to  discovering  the  basis  for  each 
person’s  decision,  this  question,  like  question  8-1,  was  intended  to  highlight  any  non- written 
pressure  to  contract  out  most  projects.  As  stated  above,  many  individuals  began  responses  to 
Q8-1  which  had  to  be  moved  to  this  question. 

Although  responses  were  varied  and  often  included  multiple  themes,  they  generally  did 
not  include  always  contracting  out  or  even  trying  to  make  80%  of  the  programs  contract  efforts. 
In  fact,  the  opposite  was  true.  Eleven  of  twenty-nine  respondents  stated  that  if  the  resources 
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were  available,  the  project  was  accomplished  internally.  Four  of  the  eleven  added  the  caveat  that 
they  would  contract  if  they  had  a  significant  amount  of  money  to  spend,  but  two  of  those  said 
they  would  try  to  do  a  concurrent  effort. 


Table  23  Unwritten  Contracting  Decision  Making  Rules 


Theme 

#  of  responses 

Do  it  in-house  if  resources  are  available  and  will 
continue  to  be  available  until  project  completion 
(unless  we  need  to  spend  money  quickly). 

11 

Do  an  analysis  to  pick  the  best  ROI. 

9 

Begin  high  risk  projects  internally. 

7 

Do  what  the  directorate  says. 

5 

Be  sure  to  spend  any  contracting  money. 

5 

Who  is  the  expert?/Does  someone  already  produce 
something  similar? 

4 

If  it  is  needed  quickly  and  can’t  slip,  do  it  internally. 

3 

Can  the  potential  contractor  be  trusted? 

2 

Can  the  product  be  easily  transitioned? 

1 

Do  we  need  to  develop  the  skill?  (if  so,  do  the  project 
internally,  but  hire  an  expert  in-house  contractor) 

1 

The  80%  contrats/20%  internal  research  rule. 

1 

The  next  highest  response  with  nine  individuals  was  to  perform  an  analysis  (cost/benefit, 
Return  on  Investment  (ROI),  make/buy  study)  to  see  which  method  is  more  cost  effective. 

Seven  of  twenty-nine  interviewees  stated  that  it  depends  upon  the  risk  or  level  of  maturity  of  a 
technology.  New  and  risky  technology  is  at  least  begun  internally  to  reduce  cost.  Five  of 
twenty-nine  individuals  claimed  they  would  follow  the  dictates  of  the  directorate,  but  only  three 
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of  those  claimed  it  was  usually  a  mandate  to  contract  out.  The  responses  are  presented  in  Table 
23. 

It  can  be  seen  that  except  for  two  excepts  (being  sure  to  spend  money  and  following  the 
80%/20%  guidance),  most  interviewees  did  not  concern  themselves  with  being  sure  to  contract 
out  a  percentage  of  the  work.  In  fact,  performing  an  external  effort  typically  seems  to  be  the 
secondary  choice. 

Oddly,  the  five  responses  concerning  following  the  directorate  were  spread  across  three 
directorates  so  this  seems  to  be  an  individual  or  possibly  branch  opinion.  As  such,  no  individual 
distinguishing  factors  seemed  to  influence  responses  to  the  question. 

Area  Nine:  General 

Q9-1)  In  your  opinion,  what  would  be  the  single  most  important  issue  in  a  debate 

between  internally  developing  and  externally  developing  systems? 

This  question  was  simply  included  to  produce  a  level  of  importance  on  the  issues  associated  with 
internal  and  external  development  by  tallying  up  the  total  number  of  each  response  theme. 
Ideally,  all  themes  presented  would  already  be  covered  in  this  thesis.  If  one  is  missing,  then  it 
would  make  a  good  candidate  to  include  in  any  follow-up  research.  The  responses  are  presented 
in  Table  24. 

The  major  themes  listed  in  the  table  were  explored  in  this  thesis.  Again,  business  case 
and  resource  availability  made  the  top  of  the  list.  All  six  interviewees  who  cited  sustained, 
accountable  resource  availability  commented  that  if  this  was  available  internal,  then  internal  was 
better.  Only  three  individuals  cited  themes  that  presented  possible  influence  to  contract  out. 

One  of  those  concerned  culture  and  the  other  two  concerned  mission  and  goals.  None  of  the 
responses  seemed  to  be  influenced  by  individual  distinguishing  factors. 
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Table  24  Important  Themes 


Theme _ #  Respondents 


Business  case  (Cost/Benefit,  ROI)  -  note:  this  involves  keeping  internal  8 

options  open  via  some  internal  projects/training. 

Sustained  resource  availability  and  accountability  for  results  (this  includes  a  6 

long  tenn  team  where  the  best  people  do  not  get  randomly  pulled  for  other 

efforts) 

Expertise/capability/integration  4 

Risk/Innovation  requirement  (evolutionary  =  contractor;  revolutionary  =  3 

government  effort) 

Transition/Sustainment  2 

Mission/Goal  (if  ARFL  rule  is  80/20,  then  we  should  do  80/20;  if  AFRL  2 

wants  to  increase  in-house  expertise,  then  that  should  be  the  rule) 

Avoiding  a  onc-sizc-fits-all  approach/balance  (the  proper  amount  of  both  2 

internal  and  external  to  get  the  best  product) 

Functionality  required  (does  it  need  the  full  acquisition  cycle  or  is  it  a  focused  1 

quick-tum  need) 

Culture  (current  culture  dictates  external  is  better)  1 

Good,  competent  program  managers  (if  they  really  know  and  understand  the  1 

system,  they  can  make  the  right  decisions) _ 


Q9-2)  Do  you  have  any  other  comments  on  the  topic? 

This  final  question  was  included  to  capture  any  missed  topics  and  inquiries.  Again,  all  responses 
should  already  be  covered  in  this  thesis.  If  one  is  missing,  it  would  make  a  good  candidate  to 
include  in  any  follow-up  research. 

First,  three  individuals  claimed  that  everybody  thinks  it  is  cheaper  to  contract  and  that 
many  large  decisions  are  based  on  this  premise.  Before  completing  this  interview  process,  the 
author  would  have  agreed  with  this  sentiment.  However,  considering  the  responses  from 
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interviewees  to  questions  throughout  the  interview  process,  it  seems  that  most  individuals  (at 
least  those  within  the  AFRL  laboratories)  understand  that  external  development  does  not  always 
save  money. 

Three  individuals  also  discussed  extremes,  which  was  a  major  concern  of  this  thesis.  One 
commented  that  “statements  like  ‘We  will  outsource  all  our  spares’  are  nearsighted.”  Another 
claimed  that  extremes  on  either  side  are  ridiculous.  Basically,  the  themes  agreed  with 
assumption  number  three  of  this  thesis:  there  is  no  one  best  acquisition  method  so  all  options 
should  be  considered. 

Two  individuals  discussed  speed  related  to  customer  requirements  and  spiral  acquisition. 
One  claimed  that:  “The  user  always  wants  the  perfect  100%  solution  immediately  and  if  they 
can’t  get  it,  they  don’t  want  to  wait.  They  need  to  accept  a  partial  solution  ASAP.  For  instance, 
basic  military  tactics  should  dictate  that  the  possibility  of  loosing  three  people  in  the  plane  to 
save  200  on  the  ground  should  be  worth  accepting  the  80%  solution  until  something  better  can  be 
developed.”  The  process  was  also  described  as  waterfall  acquisition  rather  than  spiral 
acquisition.  Although  this  scenario  was  not  directly  covered  in  the  thesis,  it  does  affect  speed, 
and,  according  to  respondents,  internal  products  can  be  developed  rapidly  but  have  difficulty 
transitioning.  Perhaps  war  fighters  should  accept  prototypes  until  full  production  is  initiated  to 
increase  speed  to  the  field. 

Two  individuals  addressed  transition.  Neither  of  these  statements  involved  separation  of 
funding  streams  or  lack  of  a  push  process  as  an  issue  so  they  are  outside  the  scope  of  this  thesis, 
but  are  included  here  for  infonnation.  They  are:  “Transition  planning  doesn’t  take  place  because 
nobody  wants  to  hand  off  their  program.  Everybody  wants  to  be  the  hero  rather  than  do  what 
they  are  good  at.”  and  “Transition  would  work  better  if  the  laboratories  were  part  of  the  SPOs  so 
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that  a  plan  for  product  use  is  in  place  at  the  beginning.”  This  second  statement  could  be 
stretched  to  conclude  that  internal  developments  could  be  more  easily  pushed  if  laboratories 
were  a  part  of  SPOs  or  it  could  be  used  to  conclude  that  all  development  efforts  should  be  based 
on  a  pull  from  the  SPOs. 

Two  interviewees  addressed  group  dynamics,  training,  rewards,  and  teaming  efforts  as 
barriers  to  internal  development.  One  claimed  that  the  laboratory  needed  to  include  more 
teaming  efforts.  New  initiatives  like  Focused  Long  Term  Challenges  were  helping,  but  most 
innovation  and  development  requires  a  team  of  individuals.  Some  people  are  led  to  be  team 
manager  while  others  like  to  be  more  involved;  training  should  reflect  this  fact.  Forcing  the 
same  training  on  everyone  is  also  unproductive.  Some  personnel  require  extra  business  training, 
some  program  management  training,  and  others  more  in-depth  engineering  training.  The  type  of 
training  should  be  based  upon  the  needs  of  the  team  rather  than  a  random  assessment  or  coded 
position.  Another  interviewee  addressed  rewards.  The  individual  pointed  out  that  the  available 
award  system  provides  high  incentives  for  individuals  and  individual  efforts,  but  nothing  for 
groups.  Since  research  generally  occurs  in  groups,  a  way  is  needed  to  provide  incentives  to 
groups.  Training  is  addressed  in  this  thesis,  but  not  to  this  detail.  However,  both  ideas  appear  to 
be  credible  methods  to  increase  internal  development  success. 

One  individual  discussed  the  concept  of  Congressional  eannarking  and  following  the 
wishes  of  Congress.  The  basic  point  was  that  Congress  wants  the  money  to  flow  through  the 
laboratory  and  be  spent  on  contracts  providing  jobs  for  their  constituents.  The  basic  80% 
contracts/20%  internal  development  rule  was  designed  to  ensure  that  this  happens,  while  enough 
internal  development  is  continued  to  maintain  capability.  Since  this  is  directed  by  Congress,  this 
must  be  included  in  the  process.  This  concept  is  addressed  in  this  thesis  in  the  literature  review. 
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The  author  agrees  with  the  interviewee  that  earmarks  must  be  spent  as  directed,  but  disagrees 
that  DoD  laboratories  should  allocate  funding  in  a  way  that  might  degrade  military  capability 
beyond  repair  or  produce  efforts  that  take  longer  than  needed  to  provide  important  developments 
to  the  war  fighter  unless  forced  to  do  so  after  thoroughly  explaining  the  consequences. 

Summary 

This  chapter  synthesized  data  collected  based  on  exploratory  case  study  methodology 
developed  by  Yin  and  Creswell.  Creswell  spiral  method  and  pattern  matching  were  used  to 
interpret  views  and  collect  themes  into  quantifiable  results.  After  analyzing  the  data,  it  appears 
that  despite  outcries  to  the  contrary,  individuals  within  AFRL  are  typically  technically  capable, 
well  trained,  and  generally  making  internal  versus  external  development  decisions  based  upon 
sound  principles  rather  than  automatically  contracting  out  a  project.  In  fact,  it  may  be  possible 
that  that  the  opposite  is  true,  that  internal  development  is  arbitrarily  chosen  if  resources  are 
available.  Synthesis  of  the  various  themes  that  emerged  during  the  analysis  portion  should 
provide  a  solid  foundation  for  drawing  conclusions  and  making  recommendations  in  chapter  five 
concerning  internal  versus  external  acquisition  efforts. 
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V.  Conclusions  and  Recommendations 


Introduction 

The  purpose  of  this  thesis  was  to  explore  the  possibility  of  internal  laboratory 
development,  testing,  and  fielding  versus  the  traditional  system  which  encourages  customer  pull 
and  contractor  development.  Assumptions  were  made  using  literature  reviews  and  preliminary 
discussions  with  laboratory  personnel  as  a  basis.  Areas  important  to  outsourcing  were  taken 
from  commercial  and  government  information  sources  and  used  to  develop  an  interview 
instrument.  DoD  laboratory  personnel  were  then  interviewed  to  gain  insight  into  their  activities, 
thoughts,  and  actions  concerning  internal  development.  This  research  was  meant  to  ascertain  if 
DoD  laboratories  had  gone  too  far  in  only  developing  though  contract  efforts.  This  chapter 
provides  conclusions  and  recommendations  based  on  insight  provided  by  this  research.  This 
chapter  also  covers  limitations  and  recommendations  for  future  research. 

Conclusions 

First,  the  assumptions  need  to  be  addressed  by  examining  each  interviewee’s  response  to 
the  lines  of  questioning. 

Assumption  1:  Current  funding  allocations  (primarily  in  6.2,  6.3  and  6.4)  do  not  allow  internal 
agencies  to  achieve  full  participation  in  product  development. 

During  question  testing  with  the  SMEs,  questions  dealing  directly  with  funding 
allocations  were  removed  because  some  personnel  were  not  aware  of  how  the  funding  streams 
mapped  to  TRLs.  Instead  the  individual  distinguishing  characteristics  of  funding  streams  and 
TRL  range  were  compared  for  abnormalities.  In  60%  of  the  interviews,  TRL  levels  and 
technology  designations  did  not  match.  This  mismatch  provided  evidence  that  funding  streams 
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were  not  being  used  exactly  as  designed  but  were,  instead,  being  used  as  needed  to  advance 
technologies. 

In  addition,  the  twelve  individuals  that  complained  about  technology  transition  during 
interviews  were  asked  if  the  division  of  money  into  separate  funding  streams  affected  their 
efforts.  All  twelve  denied  that  this  was  a  serious  issue.  Two  individuals  noted  that  the  division 
aggravated  the  situation  and  direct  access  to  6.3b  and  6.4  funds  within  the  laboratory  would  help. 
However,  all  twelve  stated  that  if  planned  correctly,  funding  stream  divisions  did  not  affect 
transition. 

Given  these  two  pieces  of  information,  it  is  safe  to  conclude  that  Assumption  One  is 
false.  The  funding  allocations  do  not  prevent  internal  agencies  from  fully  participating  in 
product  development.  It  therefore,  stands  to  reason,  that  money  already  available  is  sufficient  to 
produce  internal  efforts,  so  any  lean  toward  external  development  must  be  due  to  another  area. 
Assumption  2:  DoD  has  not  taken  full  advantage  of push  opportunities. 

This  assumption  was  addressed  directly  within  the  interview  instrument.  Question  1-1 
specifically  asked  if  there  was  an  avenue  present  to  produce  internally  developed  ideas.  This 
question  was  further  broken  down  by  immediately  available  funding  and  funding  that  required  a 
two  year  wait  on  the  PPBS  cycle.  A  seen  in  Table  5  (from  question  1-1),  although  some  funding 
was  slow,  82%  of  interviewees  had  access  to  money  needed  to  push  internal  ideas.  The  slow 
nature  of  the  PPBS  cycle  can  be  construed  as  an  issue  for  overall  development,  but  it  does  not 
present  a  bias  against  push  systems.  Therefore,  only  17%  of  respondents  felt  there  was  no 
avenue  available  to  enact  a  requirements  push  system. 

Given  this  information,  it  is  safe  to  conclude  that  Assumption  Two  is  suspect.  DoD 
definitely  does  take  advantage  of  push  opportunities  more  than  80%  of  the  time,  but  that  still 
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leaves  opportunities  that  are  lost  due  to  lack  of  an  avenue  for  a  push  process.  Still,  opportunities 
are  available  in  sufficient  quantities  to  produce  internal  efforts,  so  any  lean  toward  external 
development  must  be  due  to  another  area. 

Assumption  3:  No  one  acquisition  method  has  proven  itself  better  than  others. 

This  assumption  was  addressed  in-depth  within  the  literature  review.  Multiple 
documents  and  occasions  demonstrated  the  truth  of  this  assumption.  Extensive  support  was 
provided  through  an  in-depth  look  at  historical  success  and  failures  of  the  acquisition  process  and 
methods.  The  section  demonstrated  that  all  modern  acquisition  methods  have  been  attempted 
throughout  our  history  and  that  no  one  method  has  proven  best.  In  fact,  history  was  ripe  with 
successes  and  failures  of  each  method  and  confirmed  the  process  in  use  today  is  just  a  more 
formalized  system  with  no  fundamentally  new  ideas  or  new  acquisition  vehicles. 

Perhaps  the  continual  loop  is  caused  by  the  constant  search  for  a  silver  bullet  that  will 
always  be  the  correct  answer.  A  chase  for  the  perfect  acquisition  solution  to  all  needs  makes  for 
easy  management,  but,  it  seems,  poor  performance.  Given  the  overwhelming  historical 
information,  it  is  safe  to  conclude  Assumption  Three  is  true.  Therefore,  in  order  to  increase  the 
chance  of  developing  a  successful  system,  DoD  should  keep  options  open  by  perusing  all 
reasonably  available  methods. 

Assumption  4:  External  development  is  the  method  of  choice  for  decision  makers 

This  assumption  was  addressed  through  multiple  questions  in  the  interviews.  Specifically 
questions  4-2,  5-2,  8- 1,8-2,  and  9-1.  Questions  4-2  and  5-2  gave  an  indication  of  the  current 
state  of  the  system.  Responses  to  question  4-2  indicated  that  almost  70%  of  laboratory  personnel 
spend  70%  or  more  of  their  time  managing  contracts  versus  conducting  internal  development 
efforts,  and  almost  one-quarter  of  laboratory  personnel  spend  over  90%  of  their  time  managing 
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contracts.  These  numbers  indicate  a  definite  lean  toward  contract  work.  Responses  to  question 
5-2  paint  a  similar  picture.  Twenty-one  of  the  twenty-nine  interviewees  stated  that  training  was 
centered  on  contract  management  rather  than  engineering.  This  is  another  indicator  that  contract 
work  is  preferred.  Overall,  these  responses  seem  to  support  the  assumption  that  external 
development  is  the  method  of  choice. 

Questions  8-1,  8-2,  and  9-1  explore  guidance  and  the  interviewee’s  personal  thoughts. 
Question  8-1  indicated  that  most  interviewees  did  not  know  of  or  did  not  follow  any  official 
guidance  regarding  when  to  contract  out  and  when  to  use  internal  resources.  This  implies  that 
any  lean  toward  contracting  must  be  due  to  personal  preference.  However,  the  responses  to  8-2 
and  9-1  indicated  the  opposite.  In  both  questions,  the  majority  of  responses  were  either  to 
consider  a  business  case  or  default  to  an  internal  effort  if  resources  are  available.  Very  few 
respondents  indicated  that  development  efforts  should  default  to  contracts  unless  they  needed  to 
spend  money  quickly.  These  responses  do  not  support  the  assumption  that  external  is  the  method 
of  choice. 

The  conflicting  evidence  of  what  actually  exists  and  what  individuals  plan  makes  it 
difficult  to  draw  a  defensible  conclusion  about  the  validity  of  Assumption  Four.  Possible 
explanations  are  that  AFRL  has  a  significant  amount  of  money  which  requires  rapid  allocation 
(which  results  in  numerous  contracting  efforts),  internal  resources  are  rarely  available  (forcing 
external  contacting  efforts),  or  employees  are  actually  performing  business  case  analysis  and  the 
results  are  often  that  the  contractor  is  cheaper.  In  any  of  these  cases,  it  seems  that  Assumption 
Four  may  be  false.  Individuals  are  thinking  about  the  decision  rather  than  automatically 
defaulting  to  external  efforts. 

Component  Questions 
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Now  conclusions  related  to  the  components  of  the  research  question  will  be  addressed. 

Which  method  involves  less  cost? 

Questions  6-1  and  6-4  directly  relate  to  this  component.  Responses  to  question  6-1 
indicate  that  two-thirds  of  respondents  think  that  it  is  cheaper  to  develop  internally.  Question  6-4 
is  the  opposite  with  more  respondents  claiming  internal  development  efforts  lead  to  higher 
production  and  sustainment  costs.  However,  the  responses  to  question  6-4  are  more  evenly  split 
with  a  margin  of  eleven  to  four  with  the  other  fourteen  respondents  claming  it  depends  upon  the 
situation  or  neither  method  is  superior.  Based  on  these  areas  it  seems  that  internal  development 
is  believed  to  usually  cost  less  overall. 

Will  concurrent  efforts  reduce  cost? 

This  component  is  addressed  in  questions  2-3,  2-4,  and  6-3.  Unfortunately,  the  sample  of 
individuals  with  actual  concurrent  experience  was  limited  to  just  thirteen  individuals;  however, 
twenty-one  individuals  still  volunteered  opinions  on  the  subject.  Responses  to  both  questions  2- 
3  and  2-4  indicate  that  concurrent  development  can  reduce  costs.  Out  of  the  twenty-one 
responses  to  question  6-3,  thirteen  claimed  the  effort  would  directly  reduce  cost.  No  one 
indicated  that  that  it  would  raise  overall  costs  because  the  potential  of  getting  a  better  product 
return  was  believed  to  be  worth  the  investment.  Based  on  these  responses,  concurrent  efforts  can 
help  reduce  cost  either  directly  or  by  producing  better  products  at  a  lower  total  cost. 

Which  method  produces  better  capability? 

This  component  included  several  subcomponents  which  will  be  addressed  individually. 
The  overall  component  addresses  capability  of  products.  A  conclusion  on  this  topic  requires 
reviewing  input  from  question  1-5,  2-3,  and  2-4  to  glean  information  on  the  results  of  internal 
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efforts.  Based  on  responses,  it  seems  that  internal  development  is  just  as  capable,  but  there  is  no 
conclusive  evidence  that  one  is  better  than  the  other. 

Can  internal  be  done? 

All  questions  within  the  area  of  feasibility  relate  directly  to  this  component.  Other  questions 
applied  as  well  and  were  used  as  checks  to  ensure  conclusions  drawn  from  feasibility  were  valid. 
The  answer  was  overwhelmingly  (82%)  yes  with  just  over  half  (52%)  indicating  they  had 
everything  they  needed  on-hand.  Eighty-six  percent  had  personally  been  involved  in  at  least  one 
internal  development  effort.  Based  on  this  information,  it  seems  internal  methodology  can  be 
done.  The  political  environment  was  the  most  often  cited  stumbling  block  preventing  internal 
development. 

Does  one  method  produce  a  better  customer  interface? 

The  area  of  customer  relations  covers  this  component.  Again,  other  questions  where  customer 
service  was  mentioned  were  used  to  validate  conclusions.  Question  3-2  was  evenly  split.  The 
other  two  questions  indicated  that  translation  was  often  required  between  a  customer  and 
contractor  and  that  it  is  easier  to  implement  customer  needs  on  internal  projects.  However,  the 
majority  of  respondents  were  not  overwhelming  for  either  question  (as  indicated  by  the  number 
of  interviewees  answering  that  it  depends  or  does  not  matter).  Based  on  the  data,  it  does  not 
appear  that  either  internal  or  external  methodology  conclusively  provides  a  better  interface. 

What  defines  personnel  capability? 

This  component  is  addressed  in  both  the  training  and  capability  areas.  As  discussed  earlier, 
responses  to  question  5-2  would  imply  that  contract  management  skills  would  indicate  personnel 
capability.  Responses  to  question  5-1  generally  implied  that  training  outside  of  contract 
management  was  self-initiated,  and  thus  these  responses  do  not  contradict  5-2.  Although  some 


90 


comments  were  made  in  question  5-3  that  engineering  training  was  important,  these  were  not 
overwhelming.  Most  responses  center  on  how  the  training  should  be  conducted  rather  than  what 
should  be  taught.  Based  on  these  three  questions,  it  could  be  concluded  that  personnel  capability 
equates  to  good  contract  management  skills. 

Responses  to  questions  4-1  and  4-2  seem  to  reinforce  this  view.  Question  4-2  indicated 
that  most  individuals  spend  the  majority  of  their  time  managing  contracts  and  4-1  indicated  that 
two-thirds  of  the  respondents  thought  that  at  least  70%  of  their  peers  were  technically  proficient. 
This  seems  to  indicate  that  technical  proficiency  is  contract  management. 

However,  in  question  4-4,  only  a  small  number  of  individuals  indicated  that  a  substantial 
portion  of  the  workforce  would  require  turnover  in  order  to  switch  to  internal  development 
efforts  which  indicates  that  most  people  responded  to  question  4-1  concerning  their  peer’s  true 
engineering  ability.  The  deciding  question  is  5-4.  Even  though  training  and  capability  seem  to 
be  centered  on  contract  management,  when  asked  if  technical  degrees  were  required,  twenty-six 
of  twenty  nine  respondents  claimed  they  were  required.  Comments  generally  centered  around 
how  important  technical  expertise  and  experience  were  to  managing  contracts.  Based  on  this,  it 
seems  capability  is  defined  based  on  an  individual’s  technical  expertise. 

Does  one  method  produce  better  personnel  capability? 

According  to  question  4-2,  respondents  overwhelmingly  (twenty-eight  of  twenty-nine)  believed 
hands-on  experience  produced  more  capable  personnel.  Based  on  this,  internal  efforts  should 
produce  more  capable  personnel. 

Based  on  the  conclusions  for  all  the  sub-components  of  Which  method  produces  better 
capability? ,  it  seems  that  internal  may  be  slightly  better  because  it  has  the  upper  hand  in 
producing  more  capable  personnel. 
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Which  provides  faster  delivery? 

The  speed  to  war  fighter  area  covers  this  question  in-depth  (specifically  questions  7-1  and 

7- 2).  Responses  to  both  questions  were  evenly  split,  indicating  that  neither  internal  nor  external 
efforts  have  a  conclusive  advantage. 

Are  there  process  problems  in  either  acquisition  method? 

The  conclusion  to  this  component  is  yes.  Question  7-3  details  the  process  problems  in 
each  method  in  table  22.  However,  once  again,  each  has  their  own  issues  leaving  one  no  better 
than  the  other. 

Are  personnel  given  any  guidance  on  the  issue? 

Although  a  few  individuals  claimed  they  followed  their  directorate’s  lead,  almost  all 
individuals  claimed  that  there  was  no  official  guidance.  This  information  comes  from  questions 

8- 1  and  8-2.  This  implies  that  there  should  be  no  pressure  to  make  arbitrary  decisions. 

However,  responses  to  question  1-4,  indicating  that  political  environment  discourages  internal 
development,  are  still  an  issue. 

Finally,  the  overall  research  question  conclusions  can  be  addressed:  Is  there  a  difference 
in  warfighter  capability  and  delivery  between  internally  and  externally  developed  small 
weapons  systems  and,  if  so,  why? 

Based  on  the  analysis  presented  in  chapter  four  and  the  conclusion  to  component 
questions  above,  it  seems  that  there  should  be  no  difference  in  war  lighter  capability  and  delivery 
between  internally  and  externally  developed  small  weapons  systems.  Therefore,  internal  and 
external  methods  should  be  equally  effective  in  providing  war  fighter  support.  Internal  agencies 
should  be  provided  the  resources  to  achieve  greater  participation  in  product  development  when  it 
makes  sense. 
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It  must  be  noted  that  there  is  a  perceived  difference  in  internal  personnel  capability. 
Specifically,  internal  efforts  are  believed  to  provide  better  personnel  capability.  This  must  be 
taken  into  account  when  making  contract  decisions.  One  might  point  out  that  according  to 
responses  to  question  4-1,  personnel  capability  is  thought  to  be  high,  in  fact,  close  to  80%.  There 
are  several  possible  reasons  for  this  including:  individuals  are  trained  enough  to  be  capable, 
individuals  are  keeping  up  capability  through  collaboration  with  the  contractor,  or  much  of  this 
capability  is  concentrated  in  older  laboratory  personnel  who  are  going  to  retire.  A  quick  look  at 
those  personnel  elgible  for  retirement  indicates  that  up  to  40%  may  be  eligible  in  the  next  five 
years.  It  would  be  worth  additional  research  to  detennine  if  non-retirement  personnel  are 
capable  so  that  a  plan  can  be  enacted  accordingly. 

Recommendations 

An  environment  needs  to  be  fostered  that  encourages  selecting  the  option  (internal  or 
external)  that  is  most  advantageous  for  the  government.  The  fact  that  the  political  environment 
is  believed  to  discourage  internal  development  is  an  issue  even  if  personnel  are  not  basing 
decision  on  this  fact.  It  is  possible  that  some  opportunities  are  lost  because  individuals  consider 
the  political  environment. 

Personnel  should  be  provided  the  resources  needed  to  perform  internal  developments 
when  it  is  advantageous  for  the  government.  This  includes  breaking  down  barriers  that  prevent 
teaming,  allowing  funds  to  more  easily  flow  between  projects,  and  allowing  internal  project 
managers  to  have  more  control  over  variables  (for  example,  who  is  assigned  to  the  team  for  how 
long).  An  “internal  contract”  plan  should  also  be  developed  to  keep  internal  efforts  on  track,  on 
time,  and  accountable  for  results. 
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Efforts  should  be  made  to  perform  more  concurrent  development  when  contractors  are 
chosen  for  projects.  The  benefits  touted  are  exceptional,  and  overwhelmingly,  individuals  who 
have  participated  in  a  concurrent  effort  think  it  produced  a  better  product  and  more  capable 
internal  personnel. 

This  issue  of  internal  personnel  capability  needs  to  be  addressed.  Specifically,  is  the  high 
80%  personnel  capability  rating  being  skewed  by  retirement  eligible  individuals?  If  this  is  the 
case,  additional  internal  and  concurrent  efforts  must  be  included  to  bolster  capability  before  the 
personnel  with  the  experience  retires. 

Finally,  transition  needs  to  be  addressed.  Rapidly  developed  internal  efforts  that  wait  in 
the  production  cue  for  two  years  do  not  benefit  the  war  fighter.  Since  transition  issues  were  not 
due  to  funding  obstacles,  possible  causes  are  outside  the  scope  of  this  thesis.  However,  several 
insightful  studies  which  may  aid  in  understanding  this  issue  are  available  (Espy,  2006). 

Recommendations  for  Further  Research 

Due  to  time  and  resources,  the  thesis  was  limited  to  Air  Force  Research  Laboratories 
rather  than  including  multiple  DoD  laboratories.  Generally,  the  results  can  be  applied  at  any 
DoD  laboratory,  but  some  comments  from  other  services  would  improve  the  validity.  One 
possible  line  of  future  research  is  to  expand  this  study  to  include  multiple  DoD  laboratories. 

With  the  information  provided  in  this  exploratory  case  study,  a  survey  instrument  could  be 
devised  that  allows  input  from  a  much  larger  group  of  individuals.  Some  suggested  approaches 
based  on  the  results  of  this  study  would  be  to  expand  the  questions  dealing  with  concurrent 
development  and  reduce  the  questions  on  speed  to  war  fighter.  This  more  detailed  and  extensive 
research  would  provide  additional  clarity  into  internal  versus  external  development  efforts. 
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Another  option  would  be  to  redesign  this  study  to  get  the  customer’s  perspective  on 
internal  development.  End  user  responses  could  be  gathered  concerning  their  impressions  of 
support,  speed,  and  capability  of  contracted  versus  internally  developed  products.  The 
acquisition  community  could  also  be  included.  The  perspective  of  a  program  management  office 
might  prove  insightful.  It  is  possible  that  this  is  an  area  in  which  the  concept  that  contracting  out 
is  always  better  is  actually  being  used.  Acquisition  personnel  feedback  combined  with  end  user 
information  and  this  case  study  would  give  an  overall  acquisition  chain  impression  of  the  speed, 
support,  and  capability  of  internal  versus  external  development,  testing,  and  fielding  efforts. 

A  final  suggestion  is  to  find  examples  of  internally  developed  products  and  externally 
developed  products  which  were  developed  for  similar  efforts.  Collect  all  the  cost, 
implementation,  effectiveness,  and  results  data  for  analysis.  With  enough  examples  and  data  a 
quantitative  cost/benefit  analysis  can  be  conducted  to  determine  if  internal  or  external  efforts 
produce  a  greater  return  on  investment.  Conclusive  evidence  that  one  is  better  than  the  other  for 
certain  projects  would  be  very  important  data  for  future  decision  makers  to  use  when 
determining  whether  to  develop  internally  or  externally. 

The  possibility  exists  to  use  this  research  in  conjunction  with  other  previous  and  future 
research  to  map  out  a  decision  tree  for  managers  for  which  programs  would  be  most  viable 
internal  and  which  programs  most  viable  external.  This  is  not  recommended.  Although  it  may 
provide  an  easy  and  quick  management  tool,  it  poses  some  of  the  same  traps  discussed 
throughout  this  study  involving  missed  opportunities  due  to  a  one-size-fits-all  tool. 

Summary 

The  purpose  of  this  study  was  to  investigate  internal  laboratory  development,  testing,  and 
fielding  versus  the  traditional  system.  It  was  found  that  internal  and  external  development  were 
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believed  to  be  equally  as  effective  in  providing  war  fighter  support  and  that  avenues  to  fund 
internal  push  efforts  were  available  and  being  utilized.  It  was  also  found  that  funding  stream 
divisions  did  not  affect  the  laboratory’s  ability  to  push  technologies  and  that  personnel  were 
making  decisions  based  on  business  needs  rather  than  assuming  external  development  was 
always  best.  As  discussed  in  the  literature  review,  much  research  has  been  done  on  outsourcing 
and  its  effects  on  organizations  and  theory  has  cycled  from  internal  to  external  throughout 
industrialized  history.  This  study  adds  another  viewpoint  to  that  research  by  concluding  that 
AFRL  does  not  seem  to  be  in  any  danger  from  swaying  too  far  in  one  direction. 
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